Add a full "how to add a component" howto (and various other doc updates) [ci skip] (#3987)
* Add a walkthrough for adding a new component from scratch. There are a lot of non-obvious things you need to do when adding a new component to this repo. I've tried to capture them all in one reference guide, including screenshots for the part where you have to mess around in the XCode GUI. There are probably parts of this that could be simplified, but let's start by writing the whole process down. * Delete the "using protobuf-encoded data over Rust FFI" howto. We have some existing components that use this approach, but any new components should use UniFFI's builtin "record" types for passing data rather than using protobuf. Thus, we don't expect anyone to actually need this howto in future. * Morph the "exposing rust code on android" howto into a simpler FAQ. The existing doc was a hybrid howto/FAQ already, and we intend for the "howto" part to be entirely replaced by "use UniFFI". So, let's salvage content that's still useful to us and put it in a deliberate FAQ. * Remove the "consuming rust components on android" howto. It's basically content-free, and the one link it has there about megazords has been dead for quite some time. This doc is obviously not providing any value in practice. * Remove the "exposing rust components to swift" howto. It's pretty meagre, and it was pretty much all about the megazord config changes, which are now covered by the general "adding a new component" guide.
|
@ -0,0 +1,121 @@
|
|||
|
||||
# Rust + Android FAQs
|
||||
|
||||
### How do I expose Rust code to Kotlin?
|
||||
|
||||
Use [UniFFI](https://mozilla.github.io/uniffi-rs/), which can produce Kotlin
|
||||
bindings for your Rust code from an interface definition file.
|
||||
|
||||
If UniFFI doesn't currently meet your needs, please [open an issue](
|
||||
https://github.com/mozilla/uniffi-rs/issues) to discuss how the tool can
|
||||
be improved.
|
||||
|
||||
As a last resort, you can make hand-written bindings from Rust to Kotlin,
|
||||
essentially manually performing the steps that UniFFI tries to automate
|
||||
for you: flatten your Rust API into a bunch of `pub extern "C"` functions,
|
||||
then use [JNA](https://github.com/java-native-access/jna) to call them
|
||||
from Kotlin. The details of how to do that are well beyond the scope of
|
||||
this document.
|
||||
|
||||
### How should I name the package?
|
||||
|
||||
Published packages should be named `org.mozilla.appservices.$NAME` where `$NAME`
|
||||
is the name of your component, such as `logins`. The Java namespace in which
|
||||
your package defines its classes etc should be `mozilla.appservices.$NAME.*`.
|
||||
|
||||
### How do I publish the resulting package?
|
||||
|
||||
Add it to `.buildconfig-android.yml` in the root of this repository.
|
||||
This will cause it to be automatically included as part of our release
|
||||
publishing pipeline.
|
||||
|
||||
### How do I know what library name to load to access the compiled rust code?
|
||||
|
||||
Assuming that you're building the Rust code as part of the application-services
|
||||
build and release process, your `pub extern "C"` API should always be available
|
||||
from a file named `libmegazord.so`.
|
||||
|
||||
### What challenges exist when calling back into Kotlin from Rust?
|
||||
|
||||
There are a number of them. The issue boils down to the fact that you need to be
|
||||
completely certain that a JVM is associated with a given thread in order to call
|
||||
java code on it. The difficulty is that the JVM can GC its threads and will not
|
||||
let rust know about it.
|
||||
|
||||
JNA can work around this for us to some extent, at the cost of some complexity.
|
||||
The approach it takes is essentially to spawn a thread for each callback
|
||||
invocation. If you are certain you’re going to do a lot of callbacks and they
|
||||
all originate on the same thread, you can have them all run on a single thread
|
||||
by using the [`CallbackThreadInitializer`](
|
||||
https://java-native-access.github.io/jna/4.2.1/com/sun/jna/CallbackThreadInitializer.html).
|
||||
|
||||
With the help of JNA's workarounds, calling back from Rust into Kotlin isn’t too bad
|
||||
so long as you ensure that Kotlin cannot GC the callback while rust code holds onto it
|
||||
(perhaps by stashing it in a global variable), and so long as you can either accept the overhead of extra threads being instantiated on each call or are willing to manage
|
||||
the threads explicitly.
|
||||
|
||||
Note that the situation would be somewhat better if we used JNI directly (and
|
||||
not JNA), but this would cause us to need to generate different Rust FFI code for
|
||||
Android than for iOS.
|
||||
|
||||
Ultimately, in any case where there is an alternative to using a callback, you
|
||||
should probably pursue that alternative.
|
||||
|
||||
For example if you're using callbacks to implement async I/O, it's likely better to
|
||||
move to doing a blocking call, and have the calling code dispatch it on a background
|
||||
thread. It’s very easy to run such things on a background thread in Kotlin, is in line
|
||||
with the Android documentation on JNI usage, and in our experience is vastly simpler
|
||||
and less painful than using callbacks.
|
||||
|
||||
(Of course, not every case is solvable like this).
|
||||
|
||||
### Why are we using JNA rather than JNI, and what tradeoffs does that involve?
|
||||
|
||||
We get a couple things from using JNA that we wouldn't with JNI.
|
||||
|
||||
1. We are able to use the same Rust FFI code on all platforms. If we used JNI we'd
|
||||
need to generate an Android-specific Rust FFI crate that used the JNI APIs, and
|
||||
a separate Rust FFI crate for exposing to Swift.
|
||||
|
||||
2. JNA provides a mapping of threads to callbacks for us, making callbacks over
|
||||
the FFI possible. That said, in practice this is still error prone, and easy
|
||||
to misuse/cause memory safety bugs, but it's required for cases like logging,
|
||||
among others, and so it is a nontrivial piece of complexity we'd have to
|
||||
reimplement.
|
||||
|
||||
However, it comes with the following downsides:
|
||||
|
||||
1. JNA has bugs. In particular, its not safe to use bools with them, it thinks
|
||||
they are 32 bits, when on most platforms (every platform Rust supports) they
|
||||
are 8 bits. They've been unwilling to fix the issue due to it breaking
|
||||
backwards compatibility (which is... somewhat fair, there is a lot of C89
|
||||
code out there that uses `bool` as a typedef for a 32-bit `int`).
|
||||
2. JNA makes it really easy to do the wrong thing and have it work but corrupt
|
||||
memory. Several of the caveats around this are documented in the
|
||||
[`ffi_support` docs](https://docs.rs/ffi-support/*/ffi_support/), but a
|
||||
major one is when to use `Pointer` vs `String` (getting this wrong will
|
||||
often work, but may corrupt memory).
|
||||
|
||||
We aim to avoid triggering these bugs by auto-generating the JNA bindings
|
||||
rather than writing them by hand.
|
||||
|
||||
### How do I debug Rust code with the step-debugger in Android Studio
|
||||
|
||||
1. Uncomment the `packagingOptions { doNotStrip "**/*.so" }` line from the
|
||||
build.gradle file of the component you want to debug.
|
||||
2. In the rust code, either:
|
||||
1. Cause something to crash where you want the breakpoint. Note: Panics
|
||||
don't work here, unfortunately. (I have not found a convenient way to
|
||||
set a breakpoint to rust code, so
|
||||
`unsafe { std::ptr::write_volatile(0 as *const _, 1u8) }` usually is
|
||||
what I do).
|
||||
2. If you manage to get an LLDB prompt, you can set a breakpoint using
|
||||
`breakpoint set --name foo`, or `breakpoint set --file foo.rs --line 123`.
|
||||
I don't know how to bring up this prompt reliably, so I often do step 1 to
|
||||
get it to appear, delete the crashing code, and then set the
|
||||
breakpoint using the CLI. This is admittedly suboptimal.
|
||||
3. Click the Debug button in Android Studio, to display the "Select Deployment
|
||||
Target" window.
|
||||
4. Make sure the debugger selection is set to "Both". This tends to unset
|
||||
itself, so make sure.
|
||||
5. Click "Run", and debug away.
|
|
@ -0,0 +1,193 @@
|
|||
# Adding a new component to Application Services
|
||||
|
||||
Each component in the Application Services repo has three parts (the Rust code,
|
||||
the Kotlin wrapper, and the Swift wrapper) so there are quite a few moving
|
||||
parts involved in adding a new component. This is a rapid-fire list of all
|
||||
the things you'll need to do if adding a new component from scratch.
|
||||
|
||||
## The Rust Code
|
||||
|
||||
Your component should live under `./components` in this repo.
|
||||
Use `cargo new --lib ./components/<your_crate_name>`to create a new library crate,
|
||||
and please try to avoid using hyphens in the crate name.
|
||||
|
||||
See the [Guide to Building a Rust Component](./building-a-rust-component.md) for general
|
||||
advice on designing and structuring the actual Rust code, and follow the
|
||||
[Dependency Management Guidelines](../dependency-management.md) if your crate
|
||||
introduces any new dependencies.
|
||||
|
||||
Use [UniFFI](https://mozilla.github.io/uniffi-rs/) to define how your crate's
|
||||
API will get exposed to foreign-language bindings. By convention, put the interface
|
||||
definition file at `./components/<your_crate_name>/<your_crate_name>.udl`. Use
|
||||
the `builtin-bindgen` feature of UniFFI to simplify the build process, by
|
||||
putting this in your `Cargo.toml`:
|
||||
|
||||
```
|
||||
[build-dependencies]
|
||||
uniffi_build = { version = "<latest version here>", features=["builtin-bindgen"] }
|
||||
```
|
||||
|
||||
Include your new crate in the application-services workspace, by adding
|
||||
it to the `members` and `default-members` lists in the `Cargo.toml` at
|
||||
the root of the repository.
|
||||
|
||||
In order to be published to consumers, your crate must be included in the
|
||||
["megazord"](../design/megazords.md) crate for each target platform:
|
||||
|
||||
* For Android, add it as a dependency in `./megazords/full/Cargo.toml` and
|
||||
add a `pub use <your_crate_name>` to `./megazords/full/src/lib.rs`.
|
||||
* For iOS, add it as a dependency in `./megazords/ios/rust/Cargo.toml` and
|
||||
add a `pub use <your_crate_name>` to `./megazords/ios/rust/src/lib.rs`.
|
||||
|
||||
Run `cargo check -p <your_crate_name>` in the repository root to confirm that
|
||||
things are configured properly. This will also have the side-effect of updating
|
||||
`Cargo.lock` to contain your new crate and its dependencies.
|
||||
|
||||
|
||||
## The Kotlin Bindings
|
||||
|
||||
Make a `./components/<your_crate_name>/android` subdirectory to contain
|
||||
Kotlin- and Android-specific code. This directory will contain a gradle
|
||||
project for building your Kotlin bindings.
|
||||
|
||||
Copy the `build.gradle` file from `./components/crashtest/android/` into
|
||||
your own component's directory, and edit it to replace the references to
|
||||
`crashtest.udl` with your own component's `.udl` file.
|
||||
|
||||
Create a file `./components/<your_crate_name>/uniffi.toml` with the
|
||||
following contents:
|
||||
|
||||
```
|
||||
[bindings.kotlin]
|
||||
package_name = "mozilla.appservices.<your_crate_name>"
|
||||
cdylib_name = "megazord"
|
||||
```
|
||||
|
||||
Create a file `./components/<your_crate_name>/android/src/main/AndroidManifest.xml`
|
||||
with the following contents:
|
||||
|
||||
```
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
|
||||
package="org.mozilla.appservices.<your_crate_name>" />
|
||||
```
|
||||
|
||||
In the root of the repository, edit `.buildconfig-android.yml`to add
|
||||
your component's metadata. This will cause it to be included in the
|
||||
gradle workspace and in our build and publish pipeline. Check whether
|
||||
it builds correctly by running:
|
||||
* `./gradlew <your_crate_name>:assembleDebug`
|
||||
|
||||
You can include hand-written Kotlin code alongside the automatically
|
||||
generated bindings, by placing `.kt`` files in a directory named:
|
||||
* `./android/src/test/java/mozilla/appservices/<your_crate_name>/`
|
||||
|
||||
You can write Kotlin-level tests that consume your component's API,
|
||||
by placing `.kt`` files in a directory named:
|
||||
* `./android/src/test/java/mozilla/appservices/<your_crate_name>/`.
|
||||
|
||||
So you would end up with a directory structure something like this:
|
||||
|
||||
* `components/<your_crate_name>/`
|
||||
* `Cargo.toml`
|
||||
* `uniffi.toml`
|
||||
* `src/`
|
||||
* Rust code here.
|
||||
* `android/`
|
||||
* `build.gradle`
|
||||
* `src/`
|
||||
* `main/`
|
||||
* `AndroidManifest.xml`
|
||||
* `java/mozilla/appservices/<your_crate_name>/`
|
||||
* Hand-written Kotlin code here.
|
||||
* `test/java/mozilla/appservices/<your_crate_name>/`
|
||||
* Kotlin test-cases here.
|
||||
|
||||
Run your component's Kotlin tests with `./gradlew <your_crate_name>:test`
|
||||
to confirm that this is all working correctly.
|
||||
|
||||
|
||||
## The Swift Bindings
|
||||
|
||||
Make a `./components/<your_crate_name>/ios` subdirectory to contain
|
||||
Swift- and iOS-specific code. The UniFFI-generated swift bindings will
|
||||
be written to a subdirectory named `Generated`.
|
||||
|
||||
You can include hand-written Swift code alongside the automatically
|
||||
generated bindings, by placing `.swift` files in a directory named:
|
||||
`./ios/<your_crate_name>/`.
|
||||
|
||||
So you would end up with a directory structure something like this:
|
||||
|
||||
* `components/<your_crate_name>/`
|
||||
* `Cargo.toml`
|
||||
* `uniffi.toml`
|
||||
* `src/`
|
||||
* Rust code here.
|
||||
* `ios/`
|
||||
* `<your_crate_name>/`
|
||||
* Hand-written Swift code here.
|
||||
* `Generated/`
|
||||
* Generated Swift code will be written into this directory.
|
||||
|
||||
|
||||
Edit `megazords/ios/MozillaAppServices.h` and add an import line for your component,
|
||||
like:
|
||||
|
||||
```
|
||||
#import "uniffi_<your_crate_name>_Bridging-Header.h"
|
||||
```
|
||||
|
||||
You will then need to add your component into the iOS ["megazord"](../design/megazords.md)
|
||||
XCode project, which can only really by done using the XCode application,
|
||||
which can only really be done if you're on a Mac.
|
||||
|
||||
Open `megazords/ios/MozillaAppServices.xcodeproj` in XCode.
|
||||
|
||||
In the Project navigator, add a new Group for your new component, pointing to
|
||||
the `./ios/` directory you created above. Add the following entries to the Group:
|
||||
* The `.udl` file for you component, from `../src/<your_crate_name>.udl`.
|
||||
* Any hand-written `.swift `files for your component
|
||||
* A sub-group named "Generated", pointing to the `./Generated/` subdirectory, and
|
||||
containing entries for the files generated by UniFFI:
|
||||
* `<your_crate_name>.swift`
|
||||
* `uniffi_<your_crate_name>-Bridging-Header.h`
|
||||
|
||||
The result should look something like this:
|
||||
|
||||
![Screenshot of XCode Project Navigator](./img/xcode_add_component_1.png)
|
||||
|
||||
Click on the top-level "MozillaAppServices" project in the navigator,
|
||||
then go to "Build Phases" and add `<your_crate_name>.udl` to the list
|
||||
of "Compile Sources". This will trigger an XCode Build Rule that generates
|
||||
the Swift bindings automatically. Also include any hand-written `.swift` files
|
||||
in this list.
|
||||
|
||||
The result should look something like this:
|
||||
|
||||
![Screenshot of XCode Compile Sources list](./img/xcode_add_component_2.png)
|
||||
|
||||
In the same "Build Phases" screen, under the "Headers" section, add `uniffi_<your_crate_name>-Bridging-Header.h` to the list of Public headers.
|
||||
The result should look something like this:
|
||||
|
||||
![Screenshot of XCode Headers list](./img/xcode_add_component_3.png)
|
||||
|
||||
Build the project in XCode to check whether that all worked correctly.
|
||||
|
||||
To add Swift tests for your component API, create them in a file under
|
||||
`megazords/ios/MozillaAppServicesTests/`. Use this syntax to import
|
||||
your component's bindings from the compiled megazord:
|
||||
|
||||
```
|
||||
@testable import MozillaAppServices
|
||||
```
|
||||
|
||||
In XCode, navigate to the `MozillaAppServicesTests` Group and add your
|
||||
new test file as an entry. Select the corresponding target, click on
|
||||
"Build Phases", and add your test file to the list of "Compile Sources".
|
||||
The result should look something like this:
|
||||
|
||||
![Screenshot of XCode Test Setup](./img/xcode_add_component_4.png)
|
||||
|
||||
Use the XCode Test Navigator to run your tests and check whether
|
||||
they're passing.
|
||||
|
|
@ -21,6 +21,12 @@ To repeat with emphasis - **please consider this a living document**.
|
|||
|
||||
We think components should be structured as described here.
|
||||
|
||||
## We build libraries, not frameworks
|
||||
|
||||
Think of building a "library", not a "framework" - the application should be in
|
||||
control and calling functions exposed by your component, not providing functions
|
||||
for your component to call.
|
||||
|
||||
## The "store" is the "entry-point"
|
||||
|
||||
[Note that some of the older components use the term "store" differently; we
|
||||
|
|
|
@ -1,14 +0,0 @@
|
|||
## Guide to Consuming Rust Components on Android
|
||||
|
||||
Welcome!
|
||||
|
||||
It's great that you want to include our Rust Components into your Android app,
|
||||
but we haven't written this guide yet.
|
||||
|
||||
In fact we're still deciding whether this guide is even necessary. For now you
|
||||
probably want to find the corresponding component over in
|
||||
[android-components](https://github.com/mozilla-mobile/android-components/) and
|
||||
consume it from there.
|
||||
|
||||
You might also want to read up on [consuming megazord
|
||||
libraries](https://mozilla.github.io/application-services/docs/applications/consuming-megazord-libraries.html).
|
|
@ -1,182 +0,0 @@
|
|||
|
||||
# How to expose your Rust Component to Kotlin
|
||||
|
||||
Welcome!
|
||||
|
||||
It's great that you've built a Rust Component and want to expose it to Koltin,
|
||||
but we haven't written a complete guide yet.
|
||||
|
||||
Please share your :+1: on [the relevant github
|
||||
issue](https://github.com/mozilla/application-services/issues/599) to let us
|
||||
know that you wanted it.
|
||||
|
||||
In the meantime, here are some preliminary notes.
|
||||
|
||||
## Option 1: Autogenerated bindings with UniFFI
|
||||
|
||||
We have been working on a tool called [UniFFI](https://github.com/mozilla/uniffi-rs/) to automatically
|
||||
generate FFI bindings from the Rust code. It's probably worth a look for your use case, but
|
||||
is still quite new and may not support everything you need.
|
||||
|
||||
The [autofill](/components/autofill) component provides a useful example, and
|
||||
the [UniFFI tutorial](https://mozilla.github.io/uniffi-rs/Getting_started.html)
|
||||
walks through the steps involved.
|
||||
|
||||
## Option 2: Hand-written bindings
|
||||
|
||||
If UniFFI doesn't currently meet your needs, you may need to write your own FFI
|
||||
and bindings layer.
|
||||
|
||||
### High-level overview.
|
||||
|
||||
The [logins](/components/logins) component provides a useful example. We assume
|
||||
you have written a nice core of Rust code in a [./src/](/components/logins/src)
|
||||
directory for your component.
|
||||
|
||||
First, you will need to flatten the Rust API into a set of FFI bindings,
|
||||
conventionally in a [./ffi/](/components/logins/ffi) directory. Use the
|
||||
[ffi_support](https://docs.rs/ffi-support/0.1.3/ffi_support/) crate to help make
|
||||
this easier, which will involve implementing some traits in the core rust code.
|
||||
Consult the crate's documentation for tips and gotchas.
|
||||
|
||||
Next, you will need to write Kotlin code that consumes that FFI,
|
||||
conventionally in a [./android/](/components/logins/android) directory. This
|
||||
code should use [JNA](https://github.com/java-native-access/jna) to load the
|
||||
compiled rust code via shared library and expose it as a nice safe ergonomic
|
||||
Kotlin API.
|
||||
|
||||
It seems likely that we could provide a useful template here to get you started.
|
||||
But we haven't yet.
|
||||
|
||||
Finally, you will need to add your package into the
|
||||
[android-components](https://github.com/mozilla-mobile/android-components) repo
|
||||
via some undocumented process that starts by asking in the rust-components
|
||||
channel on slack.
|
||||
|
||||
### How should I name the resulting package?
|
||||
|
||||
Published packages should be named `org.mozilla.appservices.$NAME` where `$NAME`
|
||||
is the name of your component, such as `logins`. The Java namespace in which
|
||||
your package defines its classes etc should be `mozilla.appservices.$NAME.*`.
|
||||
|
||||
### How do I publish the resulting package?
|
||||
|
||||
Great question! We should write or link to an answer here.
|
||||
|
||||
### How do I know what library name to load to access the compiled rust code?
|
||||
|
||||
Great question! We should write or link to an answer here.
|
||||
|
||||
### Why can’t we use cbindgen or an alternative C binding generator?
|
||||
|
||||
We could, but it wouldn’t save us that much. cbindgen would automate the process
|
||||
of turning the ffi crate's source into the equivalent C header file, however it
|
||||
wouldn’t help with the kotlin code (although one could imagine something that
|
||||
automated generation of the kotlin bindings file). In particular, it wouldn’t
|
||||
help us produce the ffi crate’s code, or the wrappers that make the bindings
|
||||
safe (this is most of the work). It would be nice to automate the process, but
|
||||
it hasn’t been particularly error prone to update it by hand so far.
|
||||
|
||||
It’s also worth noting that the ffi crate’s source doesn’t have the ownership
|
||||
information encoded in it with regards to strings (they’re all pointers to
|
||||
c_char), which makes generating kotlin bindings harder, since the kotlin
|
||||
bindings need to take/return Pointer for strings owned by Rust, and String for
|
||||
strings owned by Kotlin. This is because when we later release the string owned
|
||||
by rust, it must be the same pointer we were given earlier (If you pass String,
|
||||
JNA allocates temporary memory for the call, passes you it, and releases it
|
||||
afterwards).
|
||||
|
||||
(The solution to this would likely be something like wasm_bindgen, where you
|
||||
annotate your Rust source. The tool would then spit out both the FFI crate, and
|
||||
the bulk of the Kotlin/Swift APIs. This would be a lot of work, but it would be
|
||||
cool to have someday).
|
||||
|
||||
cbindgen doesn’t always work seamlessly. It’s a standalone parser of rust code,
|
||||
not a rustc plugin -- it doesn’t always understand your rust library like the
|
||||
compiler does. A number of these issues cropped up when autopush used it: e.g.
|
||||
it being unable to parse newer rust code or having buggy handling of certain
|
||||
rust syntax.
|
||||
|
||||
### What design principles should inform my component API?
|
||||
|
||||
Many, most of which aren't written down yet. This is an incomplete list:
|
||||
|
||||
* Avoid callbacks where possible, in favour of simple blocking calls.
|
||||
* Think of building a "library", not a "framework"; the application should be in
|
||||
control and calling functions exposed by your component, not providing
|
||||
functions for your component to call.
|
||||
|
||||
### What challenges exist when calling back into Kotlin from Rust?
|
||||
|
||||
There are a number of them. The issue boils down to the fact that you need to be
|
||||
completely certain that a JVM is associated with a given thread in order to call
|
||||
java code on it. The difficulty is that the JVM can GC its threads and will not
|
||||
let rust know about it. JNA can work around this for us to some extent, however
|
||||
there are difficulties.
|
||||
|
||||
The approach it takes is essentially to spawn a thread for each callback
|
||||
invocation. If you are certain you’re going to do a lot of callbacks and they
|
||||
all originate on the same thread, you can tell it to cache these.
|
||||
|
||||
Calling back from Rust into Kotlin isn’t too bad so long as you ensure the
|
||||
callback can not be GCed while rust code holds onto it, and you can either
|
||||
accept the overhead of extra threads being instantiated on each call, or you can
|
||||
ensure that it only happens from a single thread.
|
||||
|
||||
Note that the situation would be somewhat better if we used JNI directly (and
|
||||
not JNA), but this would cause us to need to write two versions of each ffi
|
||||
crate, one for iOS, and one for Android.
|
||||
|
||||
Ultimately, in any case where you can reasonably move to making something a
|
||||
blocking call, do so. It’s very easy to run such things on a background thread
|
||||
in Kotlin. This is in line with the Android documentation on JNI usage, and my
|
||||
own experience. It’s vastly simpler and less painful this way.
|
||||
|
||||
(Of course, not every case is solvable like this).
|
||||
|
||||
### Why are we using JNA rather than JNI, and what tradeoffs does that involve?
|
||||
|
||||
We get a couple things from using JNA that we wouldn't with JNI.
|
||||
|
||||
1. We are able to write a *single* FFI crate. If we used JNI we'd need to write
|
||||
one FFI that android calls, and one that iOS calls.
|
||||
|
||||
2. JNA provides a mapping of threads to callbacks for us, making callbacks over
|
||||
the FFI possible. That said, in practice this is still error prone, and easy
|
||||
to misuse/cause memory safety bugs, but it's required for cases like logging,
|
||||
among others, and so it is a nontrivial piece of complexity we'd have to
|
||||
reimplement.
|
||||
|
||||
However, it comes with the following downsides:
|
||||
|
||||
1. JNA has bugs. In particular, its not safe to use bools with them, it thinks
|
||||
they are 32 bits, when on most platforms (every platform Rust supports) they
|
||||
are 8 bits. They've been unwilling to fix the issue due to it breaking
|
||||
backwards compatibility (which is... somewhat fair, there is a lot of C89
|
||||
code out there that uses `bool` as a typedef for a 32-bit `int`).
|
||||
2. JNA makes it really easy to do the wrong thing and have it work but corrupt
|
||||
memory. Several of the caveats around this are documented in the
|
||||
[`ffi_support` docs](https://docs.rs/ffi-support/*/ffi_support/), but a
|
||||
major one is when to use `Pointer` vs `String` (getting this wrong will
|
||||
often work, but may corrupt memory).
|
||||
|
||||
### How do I debug Rust code with the step-debugger in Android Studio
|
||||
|
||||
1. Uncomment the `packagingOptions { doNotStrip "**/*.so" }` line from the
|
||||
build.gradle file of the component you want to debug.
|
||||
2. In the rust code, either:
|
||||
1. Cause something to crash where you want the breakpoint. Note: Panics
|
||||
don't work here, unfortunately. (I have not found a convenient way to
|
||||
set a breakpoint to rust code, so
|
||||
`unsafe { std::ptr::write_volatile(0 as *const _, 1u8) }` usually is
|
||||
what I do).
|
||||
2. If you manage to get an LLDB prompt, you can set a breakpoint using
|
||||
`breakpoint set --name foo`, or `breakpoint set --file foo.rs --line 123`.
|
||||
I don't know how to bring up this prompt reliably, so I often do step 1 to
|
||||
get it to appear, delete the crashing code, and then set the
|
||||
breakpoint using the CLI. This is admittedly suboptimal.
|
||||
3. Click the Debug button in Android Studio, to display the "Select Deployment
|
||||
Target" window.
|
||||
4. Make sure the debugger selection is set to "Both". This tends to unset
|
||||
itself, so make sure.
|
||||
5. Click "Run", and debug away.
|
|
@ -1,6 +0,0 @@
|
|||
# Exposing Rust to Swift on iOS
|
||||
|
||||
* add the new FFI header include to MozillaAppServices.h. Optionally, one can add the new FFI header to the files in the project, but don't add it to the build target.
|
||||
* add the search path to the HEADER_SEARCH_PATHS in the base.xcconfig for the project to the location of the new FFI header
|
||||
* update Cargo.toml
|
||||
* update lib.rs
|
Двоичные данные
docs/howtos/img/swift-protobuf-build-rule.png
До Ширина: | Высота: | Размер: 1.0 MiB |
Двоичные данные
docs/howtos/img/swift-protobuf-framework.png
До Ширина: | Высота: | Размер: 323 KiB |
Двоичные данные
docs/howtos/img/swift-protobuf-test-setup.png
До Ширина: | Высота: | Размер: 330 KiB |
После Ширина: | Высота: | Размер: 638 KiB |
После Ширина: | Высота: | Размер: 787 KiB |
После Ширина: | Высота: | Размер: 715 KiB |
После Ширина: | Высота: | Размер: 654 KiB |
|
@ -1,539 +0,0 @@
|
|||
|
||||
# Using protobuf-encoded data over Rust FFI.
|
||||
|
||||
This assumes you already have your FFI mostly set up. If you don't that part
|
||||
should be covered by another document, which may or may not exist yet (at the time of this writing, it does not).
|
||||
|
||||
Most of this is concerned with how to do it the first time as well. If your rust
|
||||
component already is returning protobuf-encoded data, you probably just need to
|
||||
follow the examples of the other steps it takes.
|
||||
|
||||
## Rust Changes
|
||||
|
||||
1. To your main rust crate, add dependencies on the `prost` and `prost-derive` crates.
|
||||
2. Create a new file named `mylib_msg_types.proto` (well, prefix it with the
|
||||
actual name of your lib) in your main crate's `src` folder.
|
||||
|
||||
Due to annoying details of how the iOS megazord works, the name of the .proto
|
||||
file must be unique.
|
||||
|
||||
1. This file should start with
|
||||
```
|
||||
syntax = "proto2";
|
||||
package mozilla.appservices.mylib.protobuf;
|
||||
option java_package = "mozilla.appservices.mylib";
|
||||
option java_outer_classname = "MsgTypes";
|
||||
option swift_prefix = "MsgTypes_";
|
||||
option optimize_for = LITE_RUNTIME;
|
||||
```
|
||||
|
||||
The package name is going to determine where the .rs file is output.
|
||||
|
||||
2. Fill in your definitions in the rest of the file. See
|
||||
https://developers.google.com/protocol-buffers/docs/proto for examples.
|
||||
|
||||
3. In `tools/protobuf_files.toml`, add your protobuf file definition:
|
||||
```toml
|
||||
["mylib_msg_types.proto"]
|
||||
dir = "../components/mylib/src/"
|
||||
```
|
||||
4. Generate your Rust files from the .proto files using:
|
||||
|
||||
```console
|
||||
$ cargo regen-protobufs
|
||||
```
|
||||
|
||||
5. Into your main crate's lib.rs file, add something equivalent to the following:
|
||||
```rust
|
||||
pub mod msg_types {
|
||||
include!("mozilla.appservices.mylib.protobuf.rs");
|
||||
}
|
||||
```
|
||||
|
||||
This exposes the generated rust file (from the .proto file) as a
|
||||
rust module.
|
||||
|
||||
6. Open your main crates's src/ffi.rs (note: *not* ffi/src/lib.rs! We'll get
|
||||
there shortly!)
|
||||
|
||||
For each type you declare in your .proto file, first decide if you want to
|
||||
use this as the primary type to represent this data, or if you want to convert
|
||||
it from a more idiomatic Rust type into the message type when returning.
|
||||
|
||||
If it's something that exists solely to return over the FFI, or you may have
|
||||
a large number of them (or if you need to return them in an array, see the
|
||||
FAQ question on this) it *may* be best to just use the type from msg_types in your rust code.
|
||||
|
||||
We'll what you do in both cases. The parts only relevant if you are converting
|
||||
between a rust type and the protobuf start with "*(optional unless converting types)*".
|
||||
|
||||
Note that if your canonical rust type is defined in another crate, or if it's
|
||||
something like `Vec<T>`, you will need to use a wrapper. See the FAQ question
|
||||
on `Vec<T>` about this.
|
||||
|
||||
1. *(optional unless converting types)* Define the conversion between the
|
||||
idiomatic Rust type and the type produced from `msg_types`. This will
|
||||
likely look something like this:
|
||||
```rust
|
||||
impl From<HistoryVisitInfo> for msg_types::HistoryVisitInfo {
|
||||
fn from(hvi: HistoryVisitInfo) -> Self {
|
||||
Self {
|
||||
// convert url::Url to String
|
||||
url: hvi.url.into_string(),
|
||||
// Title is already an Option<String>
|
||||
title: hvi.title,
|
||||
// Convert Timestamp to i64
|
||||
timestamp: hvi.title.0 as i64,
|
||||
// Convert rust enum to i32
|
||||
visit_type: hvi.visit_type as i32,
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
2. Add a call to:
|
||||
```rust
|
||||
ffi_support::implement_into_ffi_by_protobuf!(msg_types::MyType);
|
||||
```
|
||||
|
||||
3. *(optional unless converting types)* Add a call to
|
||||
```rust
|
||||
ffi_support::implement_into_ffi_by_delegation!(MyType, msg_types::MyType);
|
||||
```
|
||||
|
||||
If `MyType` is something that you were previously returning via JSON, you need
|
||||
to remove the call to `implement_into_ffi_by_json!`, and you may also want to
|
||||
delete `Serialize` from it's `#[derive(...)]` while you're at it, unless you
|
||||
still need it.
|
||||
|
||||
7. In your ffi crate's lib.rs, make the following changes:
|
||||
|
||||
1. Any function that conceptually returns a protobuf type must now return
|
||||
`ffi_support::ByteBuffer` (if it returned via JSON before, this should be
|
||||
a change of `-> *mut c_char` to `-> ByteBuffer`).
|
||||
|
||||
Sometimes you may want to return an *optional* protobuf. Since we are returning the `RustBuffer` struct **by value** in our FFI code, it cannot be "nulled". However this structure has a `data` pointer, which will can get nulled: instead of returning a `Result<T>`, simply return `Result<Option<T>>`. In practice this means [very little changes](https://github.com/mozilla/application-services/blob/2df37a4c8c9d3b9c9159b0f80542303088027618/components/tabs/ffi/src/lib.rs#L85-L95) in Rust code.
|
||||
We have also outlined in the specific Kotlin/Swift implementations below the small changes to make to your code.
|
||||
|
||||
|
||||
2. You must add a call to
|
||||
`ffi_support::define_bytebuffer_destructor!(mylib_destroy_bytebuffer)`.
|
||||
|
||||
The name you chose for `mylib_destroy_bytebuffer` **must not** collide with the name anybody else uses for this.
|
||||
|
||||
## Kotlin Changes
|
||||
|
||||
1. Inside your component's build.gradle (e.g.
|
||||
`components/mything/android/build.gradle`, not the top level one):
|
||||
|
||||
1. Add `apply plugin: 'com.google.protobuf'` to the top of the file.
|
||||
|
||||
2. Into the `android { ... }` block, add:
|
||||
```groovy
|
||||
sourceSets {
|
||||
main {
|
||||
proto {
|
||||
srcDir '../src'
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
3. Add a new top level block:
|
||||
```groovy
|
||||
protobuf {
|
||||
protoc {
|
||||
artifact = 'com.google.protobuf:protoc:3.0.0'
|
||||
}
|
||||
plugins {
|
||||
javalite {
|
||||
artifact = 'com.google.protobuf:protoc-gen-javalite:3.0.0'
|
||||
}
|
||||
}
|
||||
generateProtoTasks {
|
||||
all().each { task ->
|
||||
task.builtins {
|
||||
remove java
|
||||
}
|
||||
task.plugins {
|
||||
javalite { }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
4. Add the following to your dependencies:
|
||||
```groovy
|
||||
implementation 'com.google.protobuf:protobuf-lite:3.0.0'
|
||||
implementation project(':native-support')
|
||||
```
|
||||
2. In the file where the foreign functions are defined, make sure that the
|
||||
function returning this type returns a `RustBuffer.ByValue` (`RustBuffer` is
|
||||
in `mozilla.appservices.support.native`).
|
||||
|
||||
Additionally, add a declaration for `mylib_destroy_bytebuffer` (the name must match what was used in the `ffi/src/lib.rs` above). This should look like:
|
||||
|
||||
```kotlin
|
||||
fun mylib_destroy_bytebuffer(v: RustBuffer.ByValue)
|
||||
```
|
||||
|
||||
3. Usage code then looks as follows:
|
||||
```kotlin
|
||||
val rustBuffer = rustCall { error ->
|
||||
MyLibFFI.INSTANCE.call_thing_returning_rustbuffer(...)
|
||||
}
|
||||
try {
|
||||
// Note: if conceptually your function returns Option<ByteBuffer>
|
||||
// from rust, you should do the following here instead:
|
||||
//
|
||||
// val message = rustBuffer.asCodedInputStream()?.let { stream ->
|
||||
// MsgTypes.SomeMessageData.parseFrom(stream)
|
||||
// }
|
||||
val message = MsgTypes.SomeMessageData.parseFrom(
|
||||
infoBuffer.asCodedInputStream()!!)
|
||||
// use `message` to produce the higher level type you want to return.
|
||||
|
||||
} finally {
|
||||
LibPlacesFFI.INSTANCE.mylib_destroy_bytebuffer(infoBuffer)
|
||||
}
|
||||
```
|
||||
|
||||
If you make any changes to the `.proto` file and want regenerate your local kotlin files, you can use:
|
||||
```
|
||||
./gradlew generateDebugProto
|
||||
```
|
||||
While being in the root directory of `application-services`.
|
||||
|
||||
## Swift
|
||||
|
||||
1. You need to install `carthage` and `swift-protobuf`, as well as `protobuf`
|
||||
(if you don't have it already). The following should do the trick if you
|
||||
use homebrew:
|
||||
```
|
||||
brew install carthage swift-protobuf protobuf
|
||||
```
|
||||
|
||||
2. Run `carthage bootstrap` from the root of this repository. This will produce a
|
||||
(gitignored) Carthage directory in this repository.
|
||||
|
||||
3. In Xcode, you need to add `Carthage/Build/iOS/SwiftProtobuf.framework`
|
||||
to your target's "Linked frameworks and libraries" list.
|
||||
- You'll need to click "Add Other" for choosing a path to be available.
|
||||
- This was hard for me to find, so see [this screenshot](./img/swift-protobuf-framework.png)
|
||||
if you are having trouble locating it.
|
||||
|
||||
4. Add `FRAMEWORK_SEARCH_PATHS = "../../../Carthage/Build/iOS"` (Note: you may need a different number of `..`s) to the `base.xcconfig`
|
||||
|
||||
5. Add your msg_types.proto file to the xcode project.
|
||||
- Do not check "copy items as needed".
|
||||
|
||||
6. Under "Build Rules", add a new build rule:
|
||||
|
||||
- Choose "Source files with names matching:" and enter `*.proto`
|
||||
|
||||
- Choose "Using custom script" and paste the following into the textarea.
|
||||
```
|
||||
protoc --proto_path=$INPUT_FILE_DIR --swift_out=$DERIVED_FILE_DIR $INPUT_FILE_PATH
|
||||
```
|
||||
- Under "Output Files", add `$(DERIVED_FILE_DIR)/$(INPUT_FILE_BASE).pb.swift`
|
||||
(with no compiler flags).
|
||||
|
||||
This should look like [this screenshot](./img/swift-protobuf-build-rule.png)
|
||||
(which should also help indicate where to find the build rules page) after
|
||||
you finish.
|
||||
|
||||
7. If you have a test target for your framework (and you should, but not all of
|
||||
our iOS code does), you need to do the following as well:
|
||||
|
||||
1. In the test target's Build phases, add SwiftProtobuf.framework to the
|
||||
"Link Binary with Libraries" phase (As in step 3, you need to "Add
|
||||
Other", and choose
|
||||
`<repo-root>/Carthage/Build/iOS/SwiftProtobuf.framework`).
|
||||
2. Add a new "Copy Files" build phase to the test target. Select the
|
||||
SwiftProtobuf.framework, and set the destination to Frameworks.
|
||||
|
||||
See [this screenshot](./img/swift-protobuf-test-setup.png) for what it should look
|
||||
like when done.
|
||||
|
||||
8. Add a declaration for the rust buffer type to your .h file:
|
||||
|
||||
Note that the name must be unique, hence the `MyLib` prefix (use your actual
|
||||
lib name, and not MyLib, of course).
|
||||
|
||||
```c
|
||||
typedef struct MyLibRustBuffer {
|
||||
int64_t len;
|
||||
uint8_t *_Nullable data;
|
||||
} MyLibRustBuffer;
|
||||
|
||||
// Note: this must be the same name you called
|
||||
// `ffi_support::define_bytebuffer_destructor!` with in
|
||||
// Rust setup step 8.
|
||||
void mylib_destroy_bytebuffer(MyLibRustBuffer bb);
|
||||
```
|
||||
|
||||
Then, your functions that return `ffi_support::ByteBuffers` in rust should
|
||||
return `MyLibRustBuffer` in the version exposed by the header, for example (from places)
|
||||
|
||||
```c
|
||||
PlacesRustBuffer bookmarks_get_by_guid(PlacesConnectionHandle handle,
|
||||
char const *_Nonnull guid,
|
||||
PlacesRustError *_Nonnull out_err);
|
||||
```
|
||||
|
||||
9. Add the following somewhere in your swift code: (TODO: Eventually we should
|
||||
figure out a way to share this)
|
||||
|
||||
```swift
|
||||
extension Data {
|
||||
init(mylibRustBuffer: MyLibRustBuffer) {
|
||||
self.init(bytes: mylibRustBuffer.data!, count: Int(mylibRustBuffer.len))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
10. Usage code then looks something like:
|
||||
|
||||
```swift
|
||||
let buffer = try MylibError.unwrap { error in
|
||||
mylib_function_returning_buffer(self.handle, error)
|
||||
}
|
||||
// Note: if conceptually your function returns Option<ByteBuffer>
|
||||
// from rust, you should do the following here:
|
||||
//
|
||||
// if buffer.data == nil {
|
||||
// return nil
|
||||
// }
|
||||
|
||||
defer { mylib_destroy_bytebuffer(buffer) }
|
||||
let msg = try! MsgTypes_MyMessageType(serializedData: Data(mylibRustBuffer: buffer))
|
||||
// use msg...
|
||||
```
|
||||
|
||||
Note: If Xcode has trouble locating the types generated from the .proto file,
|
||||
even though the build works, restarting Xcode may fix the issue.
|
||||
|
||||
# Using protobuf to pass data *into* Rust code
|
||||
|
||||
Don't pass `ffi_support::ByteBuffer`/`RustBuffer` into rust.
|
||||
It is a type for going in the other direction.
|
||||
|
||||
Instead, you should pass the data and length separately.
|
||||
|
||||
The Rust side of this looks something like this:
|
||||
|
||||
```rust
|
||||
#[no_mangle]
|
||||
pub unsafe extern "C" fn rust_fun_taking_protobuf(
|
||||
data *const u8,
|
||||
len: i32,
|
||||
error: &mut ExternError,
|
||||
) {
|
||||
// Or another call_with_blah function as needed
|
||||
ffi_support::call_with_result(error, || {
|
||||
// TODO: We should find a way to share some of this boilerplate
|
||||
assert!(len >= 0, "Bad buffer len: {}", len);
|
||||
let bytes = if len == 0 {
|
||||
// This will still fail, but as a bad protobuf format.
|
||||
&[]
|
||||
} else {
|
||||
assert!(!data.is_null(), "Unexpected null data pointer");
|
||||
std::slice::from_raw_parts(data, len as usize)
|
||||
};
|
||||
let my_thing: MyMsgType = prost::Message::decode(bytes)?;
|
||||
// Do stuff with my_thing...
|
||||
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
## Kotlin/Android
|
||||
|
||||
There are two ways of passing the data/length pairs on android. You can use
|
||||
either a `Array<Byte>` or a `Pointer`, which you can get from a "direct"
|
||||
`java.nio.ByteBuffer`. We recommend the latter, as it avoids an additional copy,
|
||||
which can be done as follows (using the `toNioDirectBuffer` our kotlin support
|
||||
library provides):
|
||||
|
||||
In Kotlin:
|
||||
|
||||
```kotlin
|
||||
// In the com.sun.jna.Library
|
||||
fun rust_fun_taking_protobuf(data: Pointer, len: Int, out: RustError.ByReference)
|
||||
|
||||
// In some your wrapper (note: `toNioDirectBuffer` is defined by our
|
||||
// support library)
|
||||
val (len, nioBuf) = theProtobufType.toNioDirectBuffer()
|
||||
rustCall { err ->
|
||||
val ptr = Native.getDirectBufferPointer(nioBuf)
|
||||
MyLib.INSTANCE.rust_fun_taking_protobuf(ptr, len, err)
|
||||
}
|
||||
```
|
||||
|
||||
Note that the `toNioDirectBuffer` helper can't return the Pointer directly, as
|
||||
it is only valid until the NIO buffer is garbage collected, and if the pointer
|
||||
were returned it would not be reachable.
|
||||
|
||||
## Swift
|
||||
|
||||
1. Make sure you've done the first 7 steps (up until you need to modify .h
|
||||
files) of the swift setup for returning protobuf data.
|
||||
|
||||
2. The function taking the protobuf should look like this in the header file:
|
||||
|
||||
```c
|
||||
void rust_fun_taking_protobuf(uint8_t const *_Nonnull data,
|
||||
int32_t len,
|
||||
MylibRustError *_Nonnull out_err);
|
||||
```
|
||||
|
||||
3. Then, the usage code from Swift would look like:
|
||||
|
||||
```swift
|
||||
var msg = MsgTypes_MyThing()
|
||||
// populate `msg` with whatever you need to send...
|
||||
|
||||
// Note: the `try!` here only fails if you failed to
|
||||
// populate a required field.
|
||||
|
||||
let data = try! msg.serializedData()
|
||||
let size = Int32(data.count)
|
||||
|
||||
try data.withUnsafeBytes { (bytes: UnsafePointer<UInt8>) in
|
||||
try MyLibError.unwrap { error in
|
||||
rust_fun_taking_protobuf(bytes, size, error)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
# FAQ
|
||||
|
||||
### What are the downsides of using types from `msg_types.proto` heavily?
|
||||
|
||||
1. It doesn't lead to particularly idiomatic Rust code.
|
||||
2. We loose the ability to enforce many type invariants that we'd like. For
|
||||
example, we cannot declare that a field holds a `Url`, and must use a
|
||||
`String` instead.
|
||||
|
||||
### I'd like to expose a function returning a `Vec<T>`.
|
||||
|
||||
If T is a type from your mylib_msg_types.proto, then this is fairly easy:
|
||||
|
||||
Don't, instead add a new msg_type that contains a repeated T field, and make
|
||||
that rust function return that.
|
||||
|
||||
Then, make so long as the new msg_type has `implement_into_ffi_by_protobuf!` and the ffi function returns a ByteBuffer, things should "Just Work".
|
||||
|
||||
---
|
||||
|
||||
Unfortunately, if T is merely *convertable* to something from mylib_msg_types.proto,
|
||||
this adds a bunch of boilerplate.
|
||||
|
||||
Say we have the following mylib_msg_types.proto:
|
||||
|
||||
```proto
|
||||
message HistoryVisitInfo {
|
||||
required string url = 1;
|
||||
optional string title = 2;
|
||||
required int64 timestamp = 3;
|
||||
required int32 visit_type = 4;
|
||||
}
|
||||
message HistoryVisitInfos {
|
||||
repeated HistoryVisitInfo infos = 1;
|
||||
}
|
||||
```
|
||||
|
||||
in src/ffi.rs, we then need
|
||||
|
||||
```rust
|
||||
// Convert from idiomatic rust HistoryVisitInfo to msg_type HistoryVisitInfo
|
||||
impl From<HistoryVisitInfo> for msg_types::HistoryVisitInfo {
|
||||
fn from(hvi: HistoryVisitInfo) -> Self {
|
||||
Self {
|
||||
url: hvi.url,
|
||||
title: hvi.title,
|
||||
timestamp: hvi.title.0 as i64,
|
||||
visit_type: hvi.visit_type as i32,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Declare a type that exists to wrap the vec (see the next question about
|
||||
// why this is needed)
|
||||
pub struct HistoryVisitInfos(pub Vec<HistoryVisitInfo>);
|
||||
|
||||
// Define the conversion between said wrapper and the protobuf
|
||||
// HistoryVisitInfos
|
||||
impl From<HistoryVisitInfos> for msg_types::HistoryVisitInfos {
|
||||
fn from(hvis: HistoryVisitInfos) -> Self {
|
||||
Self {
|
||||
infos: hvis.0
|
||||
.into_iter()
|
||||
.map(msg_types::HistoryVisitInfo::from)
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// generate the IntoFfi for msg_types::HistoryVisitInfos
|
||||
implement_into_ffi_by_protobuf!(msg_types::HistoryVisitInfos);
|
||||
// Use it to implement it for HistoryVisitInfos
|
||||
implement_into_ffi_by_delegation!(HistoryVisitInfos, msg_types::HistoryVisitInfos);
|
||||
```
|
||||
|
||||
Then, in `ffi/src/lib.rs`, where you currently return the Vec, you need to
|
||||
change it to return wrap that in main_crate::ffi::HistoryVisitInfos, something like
|
||||
|
||||
```rust
|
||||
CONNECTIONS.call_with_result(error, handle, |conn| -> places::Result<_> {
|
||||
Ok(HistoryVisitInfos(storage::history::get_visit_infos(
|
||||
conn,
|
||||
places::Timestamp(start_date.max(0) as u64),
|
||||
places::Timestamp(end_date.max(0) as u64),
|
||||
)?))
|
||||
})
|
||||
```
|
||||
|
||||
### Why is that so painful?
|
||||
|
||||
Yep. There are a few reasons for this.
|
||||
|
||||
`ffi_support` is the only one who is in a position to decide how a `Vec<T>` is
|
||||
returned over the FFI. Rust has a rule that either the trait (in this case
|
||||
`IntoFfi`) or the type (in this case `Vec`) must be implemented in the crate
|
||||
where the `impl` block happens. This is known as the orphan rule.
|
||||
|
||||
Additionally, until rust gains support for
|
||||
[specialization](https://github.com/rust-lang/rust/issues/31844), we have very
|
||||
little flexibility with how this works. We can't implement it one way for some
|
||||
kinds of T's, and another way for others (however, we can, and do, make it
|
||||
opt-in, but that's unrelated).
|
||||
|
||||
This means ffi_support is in the position of deciding how `Vec<T>` goes over the
|
||||
FFI for all T. At one point, the reasonable choice seemed to be JSON. This is
|
||||
still used fairly heavily for returning arrays of things, and so until we move
|
||||
*everything* to use protobufs, we don't really want to take that out.
|
||||
|
||||
Unfortunately even we no longer use JSON for this, the conversion between
|
||||
`Vec<T>` and the ByteBuffer has to happen through an intermediate type, due to
|
||||
the way protobuf messages work (you can't have a message that's an array, but
|
||||
you *can* have one that is a single item type which contains a repeated array),
|
||||
and it isn't clear how to make this work (it can't be an argument to a macro, as
|
||||
that would violate the orphan rule).
|
||||
|
||||
The only thing that would work is if we use the types generated by prost for more
|
||||
than just returning things over the FFI. e.g. the rust `get_visit_infos()` call would return `HistoryVisitInfos` struct that is generated from a `.proto` file.
|
||||
|
||||
#### Could this be worked around by using length-delimited protobuf messages?
|
||||
|
||||
Yes, possibly. Looking into this is something we may do in the future.
|
||||
|
||||
### Why is the module produced from .proto `msg_types` and not `ffi_types`?
|
||||
|
||||
We use `msg_types` and not e.g. `ffi_types`, since in some cases (see the next
|
||||
FAQ about returning arrays, for example) it can reduce boilerplate a lot to use
|
||||
these for returning the data to rust code directly (particularly when the rust
|
||||
API exists almost exclusively to be called from the FFI).
|
||||
|
||||
Using a name like `ffi_types`, while possibly intuitive, gives the impression
|
||||
that these types should not be used outside the FFI, and that it may even be
|
||||
unsafe to do so.
|
|
@ -4,6 +4,9 @@ All names in this project should adhere to the guidelines outlined in this docum
|
|||
|
||||
## Rust Code
|
||||
|
||||
TL;DR: do what Rust's builtin warnings and clippy lints tell you
|
||||
(and CI will fail if there are any unresolved warnings or clippy lints).
|
||||
|
||||
### Overview
|
||||
|
||||
- All variable names, function names, module names, and macros in Rust code should follow typical `snake_case` conventions.
|
||||
|
|