Bug 1648348 - Create "Writing Rust Code" docs. r=froydnj,zbraniecki,lina.

This patch includes content from the following places.

- Lina's "Getting Rusty: How to ship an XPCOM component in Firefox" slide deck.
- Zibi's "Rust <--> C/C++ FFI for newbies" gist.

It also links to Emilio's "FFI patterns #1 - Complex Rust data structures
exposed seamlessly to C++" blog post. I was going to include that content, but
it's very long, so I have omitted it for now.

Differential Revision: https://phabricator.services.mozilla.com/D81963

build/docs/rust.rst

@@ -4,9 +4,12 @@
 Including Rust Code in Firefox
 ==============================
 
-The build system has support for building, linking, and vendoring Rust crates.
+This page explains how to add, build, link, and vendor Rust crates.
-It is straightforward to take an existing Rust crate and integrate it into
+
-Firefox.
+The `code documentation <../../writing-rust-code>`_ explains how to write and
+work with Rust code in Firefox. The
+`test documentation <../../testing-rust-code>`_ explains how to test and debug
+Rust code in Firefox.
 
 Linking Rust crates into libxul
 ===============================

docs/config.yml

@@ -24,6 +24,7 @@ categories:
         - uriloader
         - widget/cocoa
         - code-quality
+        - writing-rust-code
     build_doc:
         - mach
         - tools/try

docs/testing-rust-code/index.md

@@ -1,5 +1,11 @@
 # Testing & Debugging Rust Code
 
+This page explains how to test and debug Rust code in Firefox.
+
+The [build documentation](../build/buildsystem/rust.html) explains how to add
+new Rust code to Firefox. The [code documentation](../writing-rust-code)
+explains how to write and work with Rust code in Firefox.
+
 ## Testing Mozilla crates
 
 Rust code will naturally be tested as part of system tests such as Mochitests.

docs/writing-rust-code/basics.md

@@ -0,0 +1,83 @@
+# Basics
+
+## Formatting Rust code
+
+To format all the Rust code within a directory `$DIR`, run:
+```
+./mach lint -l rustfmt --fix $DIR
+```
+
+## Using Cargo
+
+Many Cargo commands can be run on individual crates. Change into the directory
+containing the crate's `Cargo.toml` file, and then run the command with
+`MOZ_TOPOBJDIR` set appropriately. For example, to generate and view rustdocs
+for the `xpcom` crate, run these commands:
+
+```
+cd xpcom/rust/xpcom
+MOZ_TOPOBJDIR=$OBJDIR cargo doc
+cd -
+firefox target/doc/xpcom/index.html 
+```
+where `$OBJDIR` is the path to the object directory.
+
+## Using static prefs
+
+Static boolean/integer prefs can be easily accessed from Rust code. Add a
+`rust: true` field to the pref definition in
+[modules/libpref/init/StaticPrefList.yaml](https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml),
+like this:
+```yaml
+- name: my.lucky.pref
+  type: RelaxedAtomicBool
+  value: true
+  mirror: always
+  rust: true
+```
+The pref can then be accessed via the `pref!` macro, like this:
+```
+let my_lucky_pref = static_prefs::pref!("my.lucky.pref");
+```
+
+## Helper crates
+
+The following in-tree helper crates provide idiomatic support for some common patterns.
+- [nserror](https://searchfox.org/mozilla-central/source/xpcom/rust/nserror/src/lib.rs)
+reflects `nsresult` codes into Rust.
+- [nsstring](https://searchfox.org/mozilla-central/source/xpcom/rust/nsstring/src/lib.rs)
+  exposes bindings for XPCOM string types. You can use the same `ns{A,C}String`
+  types as C++ for owned strings and pass them back and forth over the
+  boundary. There is also `ns{A,C}Str` for dependent or borrowed strings.
+- [xpcom](https://searchfox.org/mozilla-central/source/xpcom/rust/xpcom/src)
+  provides multiple building blocks for a component's implementation.
+  - The `RefPtr` type is for managing reference-counted pointers.
+  - XPCOM service getters are generated by
+    [xpcom/build/Services.py](https://searchfox.org/mozilla-central/source/xpcom/build/Services.py)
+    and can be called like this:
+    ```
+    let pref_service = xpcom::services::get_PrefService();
+    ```
+  - There is also a `get_service` function that works like `do_GetService` in
+    C++, as an alternative.
+  - A set of `derive` macros help with declaring interface implementations. The
+    [docs](https://searchfox.org/mozilla-central/source/xpcom/rust/xpcom/xpcom_macros/src/lib.rs)
+    have details and examples.
+- [moz_task](https://searchfox.org/mozilla-central/source/xpcom/rust/moz_task/src/lib.rs)
+  wraps XPCOM's threading functions in order to make it easy and safe to write
+  threaded code. It has helpers for getting and creating threads, dispatching
+  async runnables, and thread-safe handles.
+- [storage](https://searchfox.org/mozilla-central/source/storage/rust/src/lib.rs)
+  is an interface to mozStorage, our wrapper for SQLite. It can wrap an
+  existing storage connection, and prepare and execute statements. This crate
+  wraps the synchronous connection API, and lets you execute statements
+  asynchronously via `moz_task`.
+- [storage_variant](https://searchfox.org/mozilla-central/source/storage/variant/src/lib.rs)
+  is for working with variants. It also provides a `HashPropertyBag` type
+  that's useful for passing hash maps over XPCOM to JS.
+
+Unfortunately, rustdocs are [not yet generated and
+hosted](https://bugzilla.mozilla.org/show_bug.cgi?id=1428139) for crates within
+mozilla-central. Therefore, the crate links shown above link to files
+containing the relevant rustdocs source where possible. However, you can
+generate docs locally using the `cargo doc` command described above.

docs/writing-rust-code/ffi.md

@@ -0,0 +1,240 @@
+# Rust/C++ interop
+
+This document describes how to use FFI in Firefox to get Rust code and C++ code to interoperate.
+
+## Transferable types
+
+Generally speaking, the more complicated is the data you want to transfer, the
+harder it'll be to transfer across the FFI boundary.
+
+Booleans, integers, and pointers cause little trouble.
+- C++ `bool` matches Rust `bool`
+- C++ `uint8_t` matches Rust `u8`, `int32_t` matches Rust `i32`, etc.
+- C++ `const T*` matches Rust `*const T`, `T*` matches Rust `*mut T`.
+
+Lists are handled by C++ `nsTArray` and Rust `ThinVec`.
+
+For strings, it is best to use the `nsstring` helper crate. Using a raw pointer
+plus length is also possible for strings, but more error-prone.
+
+If you need a hashmap, you'll likely want to decompose it into two lists (keys
+and values) and transfer them separately.
+
+Other types can be handled with tools that generate bindings, as the following
+sections describe.
+
+## Accessing C++ code and data from Rust
+
+To call a C++ function from Rust requires adding a function declaration to Rust.
+For example, for this C++ function:
+
+```
+extern "C" {
+bool UniquelyNamedFunction(const nsCString* aInput, nsCString* aRetVal) {
+  return true;
+}
+}
+```
+add this declaration to the Rust code:
+```rust
+extern "C" {
+    pub fn UniquelyNamedFunction(input: &nsCString, ret_val: &mut nsCString) -> bool;
+}
+```
+
+Rust code can now call `UniquelyNamedFunction()` within an `unsafe` block. Note
+that if the declarations do not match (e.g. because the C++ function signature
+changes without the Rust declaration being updated) crashes are likely. (Hence
+the `unsafe` block.)
+
+Because of this unsafety, for non-trivial interfaces (in particular when C++
+structs and classes must be accessed from Rust code) it's common to use
+[rust-bindgen](https://github.com/rust-lang/rust-bindgen), which generates Rust
+bindings. The documentation is
+[here](https://rust-lang.github.io/rust-bindgen/).
+
+## Accessing Rust code and data from C++
+
+A common option for accessing Rust code and data from C++ is to use
+[cbindgen](https://github.com/eqrion/cbindgen), which generates C++ header
+files. for Rust crates that expose a public C API. cbindgen is a very powerful
+tool, and this section only covers some basic uses of it.
+
+### Basics
+
+First, add suitable definitions to your Rust. `#[no_mangle]` and `extern "C"`
+are required.
+
+```rust
+#[no_mangle]
+pub unsafe extern "C" fn unic_langid_canonicalize(
+    langid: &nsCString,
+    ret_val: &mut nsCString
+) -> bool {
+    ret_val.assign("new value");
+    true
+}
+```
+
+Then, add a `cbindgen.toml` file in the root of your crate. It may look like this:
+
+```toml
+header = """/* This Source Code Form is subject to the terms of the Mozilla Public
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */"""
+autogen_warning = """/* DO NOT MODIFY THIS MANUALLY! This file was generated using cbindgen. See RunCbindgen.py */
+#ifndef mozilla_intl_locale_MozLocaleBindings_h
+#error "Don't include this file directly, instead include MozLocaleBindings.h"
+#endif
+"""
+include_version = true
+braces = "SameLine"
+line_length = 100
+tab_width = 2
+language = "C++"
+# Put FFI calls in the `mozilla::intl::ffi` namespace.
+namespaces = ["mozilla", "intl", "ffi"]
+
+# Export `ThinVec` references as `nsTArray`.
+[export.rename]
+"ThinVec" = "nsTArray"
+```
+
+Next, extend the relevant `moz.build` file to invoke cbindgen.
+
+```python
+if CONFIG['COMPILE_ENVIRONMENT']:
+    CbindgenHeader('unic_langid_ffi_generated.h',
+                   inputs=['/intl/locale/rust/unic-langid-ffi'])
+
+    EXPORTS.mozilla.intl += [
+        '!unic_langid_ffi_generated.h',
+    ]
+```
+
+This tells the build system to run cbindgen on
+`intl/locale/rust/unic-langid-ffi` to generate `unic_langid_ffi_generated.h`,
+which will be placed in `$OBJDIR/dist/include/mozilla/intl/`.
+
+Finally, include the generated header into a C++ file and call the function.
+
+```c++
+#include "mozilla/intl/unic_langid_ffi_generated.h"
+
+using namespace mozilla::intl::ffi;
+
+void Locale::MyFunction(nsCString& aInput) const {
+  nsCString result;
+  unic_langid_canonicalize(aInput, &result);
+}
+```
+
+### Complex types
+
+Many complex Rust types can be exposed to C++, and cbindgen will generate
+appropriate bindings for all `pub` types. For example:
+
+```rust
+#[repr(C)]
+pub enum FluentPlatform {
+    Linux,
+    Windows,
+    Macos,
+    Android,
+    Other,
+}
+
+extern "C" {
+    pub fn FluentBuiltInGetPlatform() -> FluentPlatform;
+}
+```
+
+```c++
+ffi::FluentPlatform FluentBuiltInGetPlatform() {
+  return ffi::FluentPlatform::Linux;
+}
+```
+
+For an example using cbindgen to expose much more complex Rust types to C++,
+see [this blog post].
+
+[this blog post]: https://crisal.io/words/2020/02/28/C++-rust-ffi-patterns-1-complex-data-structures.html
+
+### Instances
+
+If you need to create and destroy a Rust struct from C++ code, the following
+example may be helpful.
+
+First, define constructor, destructor and getter functions in Rust. (C++
+declarations for these will be generated by cbindgen.)
+
+```rust
+#[no_mangle]
+pub unsafe extern "C" fn unic_langid_new() -> *mut LanguageIdentifier {
+    let langid = LanguageIdentifier::default();
+    Box::into_raw(Box::new(langid))
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn unic_langid_destroy(langid: *mut LanguageIdentifier) {
+    drop(Box::from_raw(langid));
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn unic_langid_as_string(
+    langid: &mut LanguageIdentifier,
+    ret_val: &mut nsACString,
+) {
+    ret_val.assign(&langid.to_string());
+}
+```
+
+Next, in a C++ header define a destructor via `DefaultDelete`.
+
+```c++
+#include "mozilla/intl/unic_langid_ffi_generated.h"
+#include "mozilla/UniquePtr.h"
+
+namespace mozilla {
+
+template <>
+class DefaultDelete<intl::ffi::LanguageIdentifier> {
+ public:
+  void operator()(intl::ffi::LanguageIdentifier* aPtr) const {
+    unic_langid_destroy(aPtr);
+  }
+};
+
+}  // namespace mozilla
+```
+
+(This definition must be visible any place where
+`UniquePtr<intl::ffi::LanguageIdentifier>` is used, otherwise C++ will try to
+free the code, which might lead to strange behaviour!)
+
+Finally, implement the class.
+
+```c++
+class Locale {
+public:
+  explicit Locale(const nsACString& aLocale)
+    : mRaw(unic_langid_new()) {}
+
+  const nsCString Locale::AsString() const {
+    nsCString tag;
+    unic_langid_as_string(mRaw.get(), &tag);
+    return tag;
+  }
+
+private:
+  UniquePtr<ffi::LanguageIdentifier> mRaw;
+}
+```
+
+This makes it possible to instantiate a `Locale` object and call `AsString()`,
+all from C++ code.
+
+## Other examples
+
+For a detailed explanation of an interface in Firefox that doesn't use cbindgen
+or rust-bindgen, see [this blog post](https://hsivonen.fi/modern-cpp-in-rust/).

docs/writing-rust-code/index.md

@@ -0,0 +1,17 @@
+# Writing Rust Code
+
+This page explains how to write and work with Rust code in Firefox, with an
+emphasis on interoperation with C++ code.
+
+The [build documentation](../build/buildsystem/rust.html) explains how to add
+new Rust code to Firefox. The [test documentation](../testing-rust-code)
+explains how to test and debug Rust code in Firefox.
+
+```eval_rst
+.. toctree::
+    :titlesonly:
+    :maxdepth: 1
+    :glob:
+
+    *
+```

docs/writing-rust-code/xpcom.md

@@ -0,0 +1,123 @@
+# XPCOM components in Rust
+
+XPCOM components can be written in Rust.
+
+## A tiny example
+
+The following example shows a new type that implements `nsIObserver`.
+
+First, create a new empty crate (e.g. with `cargo init --lib`), and add the
+following dependencies in its `Cargo.toml` file.
+
+```toml
+[dependencies]
+libc = "0.2"
+nserror = { path = "../../../xpcom/rust/nserror" }
+nsstring = { path = "../../../xpcom/rust/nsstring" }
+xpcom = { path = "../../../xpcom/rust/xpcom" }
+```
+
+(The number of `../` occurrences will depend on the depth of the crate in the
+file hierarchy.)
+
+Next hook it into the build system according to the [build
+documentation](../build/buildsystem/rust.html).
+
+The Rust code will need to import some basic types. `xpcom::interfaces`
+contains all the usual `nsI` interfaces.
+
+```rust
+use libc::c_char;
+use nserror::nsresult;
+use std::sync::atomic::{AtomicBool, Ordering};
+use xpcom::{interfaces::nsISupports, RefPtr};
+```
+
+The next part declares the implementation. 
+
+```rust
+#[derive(xpcom)]
+#[xpimplements(nsIObserver)]
+#[refcnt = "atomic"]
+struct InitMyObserver {
+    ran: AtomicBool,
+}
+```
+
+It defines an initializer struct, prefixed with `Init`, with three attributes.
+- Some `derive` magic.
+- An `xpimplements` declaration naming the interface(s) being implemented.
+- The reference count type.
+
+Next, all interface methods are declared in the `impl` block as `unsafe` methods.
+
+```rust
+impl MyObserver {
+    #[allow(non_snake_case)]
+    unsafe fn Observe(
+        &self,
+        _subject: *const nsISupports,
+        _topic: *const c_char,
+        _data: *const i16,
+    ) -> nsresult {
+        self.ran.store(true, Ordering::SeqCst);
+        nserror::NS_OK
+    }
+}
+```
+
+These methods always take `&self`, not `&mut self`, so we need to use interior
+mutability: `AtomicBool`, `RefCell`, `Cell`, etc.
+
+XPCOM methods are unsafe by default, but the
+[xpcom_method!](https://searchfox.org/mozilla-central/source/xpcom/rust/xpcom/src/method.rs)
+macro can be used to clean this up. It also takes care of null-checking and
+hiding pointers behind references, lets you return a `Result` instead of an
+`nsresult,` and so on.
+
+To use this type within Rust code, do something like the following.
+
+```rust
+let observer = MyObserver::allocate(InitMyObserver {
+  ran: AtomicBool::new(false),
+});
+let rv = unsafe {
+  observer.Observe(x.coerce(),
+                   cstr!("some-topic").as_ptr(),
+                   ptr::null())
+};
+assert!(rv.succeeded());
+```
+
+The implementation has an (auto-generated) `allocate` method that takes in an
+initialization struct, and returns a `RefPtr` to the instance.
+
+`coerce` casts any XPCOM object to one of its base interfaces; in this case,
+the base interface is `nsISupports`. In C++, this would be handled
+automatically through inheritance, but Rust doesn’t have inheritance, so the
+conversion must be explicit.
+
+## Bigger examples
+
+The following XPCOM components are written in Rust.
+- [kvstore](https://searchfox.org/mozilla-central/source/toolkit/components/kvstore),
+  which exposes the LMDB key-value store (via the [Rkv
+  library](https://docs.rs/rkv)) The API is asynchronous, using `moz_task` to
+  schedule all I/O on a background thread, and supports getting, setting, and
+  iterating over keys.
+- [cert_storage](https://searchfox.org/mozilla-central/source/security/manager/ssl/cert_storage),
+  which stores lists of [revoked intermediate certificates](https://blog.mozilla.org/security/2015/03/03/revoking-intermediate-certificates-introducing-onecrl/).
+- [bookmark_sync](https://searchfox.org/mozilla-central/source/toolkit/components/places/bookmark_sync),
+  which [merges](https://mozilla.github.io/dogear) bookmarks from Firefox Sync
+  with bookmarks in the Places database.
+- [webext_storage_bridge](https://searchfox.org/mozilla-central/source/toolkit/components/extensions/storage/webext_storage_bridge),
+  which powers the WebExtension storage.sync API. It's a self-contained example
+  that pulls in a crate from application-services for the heavy lifting, wraps
+  that up in a Rust XPCOM component, and then wraps the component in a JS
+  interface. There's also some boilerplate there around adding a
+  `components.conf` file, and a dummy C++ header that declares the component
+  constructor.
+- [firefox-accounts-bridge](https://searchfox.org/mozilla-central/source/services/fxaccounts/rust-bridge/firefox-accounts-bridge),
+  which wraps the Rust Firefox Accounts client with which we eventually want to
+  replace our creaky JS implementation. It has a lot of the same patterns as
+  webext_storage_bridge.

moz.build

@@ -200,6 +200,8 @@ SPHINX_TREES['code-quality'] = 'docs/code-quality'
 
 SPHINX_TREES['testing-rust-code'] = 'docs/testing-rust-code'
 
+SPHINX_TREES['writing-rust-code'] = 'docs/writing-rust-code'
+
 SPHINX_TREES['bug-mgmt'] = 'docs/bug-mgmt'
 
 SPHINX_TREES['setup'] = 'docs/setup'