Bug 1647987 - Create Rust testing docs. r=froydnj.

Some of the testing info is from the Oxidation wiki, and the logging info is
largely from a dev-platform email by Valentin. The other parts I wrote from
scratch.

The commit also makes some small improvements to the Rust build docs.

Differential Revision: https://phabricator.services.mozilla.com/D81017

build/docs/rust.rst

@@ -13,17 +13,17 @@ Linking Rust crates into libxul
 
 Rust crates that you want to link into libxul should be listed in the
 ``dependencies`` section of
-`toolkit/library/rust/shared/Cargo.toml <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml>`_.
+`toolkit/library/rust/shared/Cargo.toml <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml>`_.
 After adding your crate, execute ``cargo update -p gkrust-shared``
 to update the ``Cargo.lock`` file. You'll also need to add an ``extern crate``
 reference to
-`toolkit/library/rust/shared/lib.rs <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_.
+`toolkit/library/rust/shared/lib.rs <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_.
 This ensures that the Rust code will be linked properly into libxul as well
 as the copy of libxul used for gtests.
 
 By default, all Cargo packages in the mozilla-central repository are part of
 the same
-`workspace <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_
+`workspace <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_
 and will share the ``Cargo.lock`` file and ``target`` directory in the root of
 the repository.  You can change this behavior by adding a path to the
 ``exclude`` list in the top-level ``Cargo.toml`` file.  You may want to do
@@ -49,7 +49,7 @@ To link Rust code into libraries other than libxul, create a directory with a
     RustLibrary('crate_name')
 
 where ``crate_name`` matches the name from the ``[package]`` section of your
-``Cargo.toml``. You can refer to `the moz.build file <https://dxr.mozilla.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/moz.build#7>`_ and `the Cargo.toml file <https://dxr.mozilla.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/Cargo.toml>`_ that are used for libxul.
+``Cargo.toml``. You can refer to `the moz.build file <https://searchfox.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/moz.build#7>`_ and `the Cargo.toml file <https://searchfox.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/Cargo.toml>`_ that are used for libxul.
 
 You can then add ``USE_LIBS += ['crate_name']`` to the ``moz.build`` file
 that defines the binary as you would with any other library in the tree.
@@ -119,6 +119,10 @@ into mozilla-central, keep the following in mind.
 - ``mach vendor rust`` will check that the licenses of all crates are suitable.
 - You should review the crate code to some degree to check that it looks
   reasonable (especially for unsafe code) and that it has reasonable tests.
+- Third-party crate tests aren't run, which means that large test fixtures will
+  bloat mozilla-central. Consider working with upstream to mark those test
+  fixtures with ``[package] exclude = ...`` as described
+  `here <https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields>`_.
 - Other than that, there is no formal sign-off procedure, but one may be added
   in the future.
 
@@ -148,3 +152,9 @@ Then, make the local changes to the crate.
 
 Finally, make sure you don't accidentally land the changes to the crate or the
 ``Cargo.lock`` file.
+
+For an example of a more complex workflow involving a third-party crate, see
+`mp4parse-rust/README.md <https://searchfox.org/mozilla-central/source/media/mp4parse-rust/README.md>`_.
+It describes the workflow for a crate that is hosted on GitHub, and for which
+changes are made via GitHub pull requests, but all pull requests must also be
+tested within mozilla-central before being merged.

docs/config.yml

@@ -39,6 +39,7 @@ categories:
         - testing/perfdocs
         - testing/perftest
         - tools/code-coverage
+        - testing-rust-code
     l10n_doc:
         - intl
         - l10n

docs/testing-rust-code/index.md

@@ -0,0 +1,117 @@
+# Testing & Debugging Rust Code
+
+## Testing Mozilla crates
+
+Rust code will naturally be tested as part of system tests such as Mochitests.
+This section describes the two methods for unit testing of individual Rust
+crates. Which method should be used depends on the circumstances.
+
+### Rust tests
+
+If a Mozilla crate has "normal" Rust tests (i.e. `#[test]` functions that run
+with `cargo test`), you can add the crate's name to `RUST_TESTS` in
+[toolkit/library/rust/moz.build](https://searchfox.org/mozilla-central/source/toolkit/library/rust/moz.build).
+(Cargo features can be activated for Rust tests by adding them to
+`RUST_TEST_FEATURES` in the same file.)
+
+Rust tests are run with `./mach rusttests`. They run on automation in a couple
+of `rusttests` jobs, but not on all platforms.
+
+Rust tests have one major restriction: they cannot link against Gecko symbols.
+Therefore, Rust tests cannot be used for crates that use Gecko crates like
+`nsstring` and `xpcom`.
+
+It's also possible to use `RUST_TESTS` in a different `moz.build` file. See
+`testing/geckodriver/moz.build` and the [geckodriver testing docs] for an
+example.
+
+[geckodriver testing docs]: ../testing/geckodriver/Testing.html
+
+### GTests
+
+Another way to unit test a Mozilla crate is by writing a GTest that uses FFI to
+call into Rust code. This requires the following steps.
+- Create a new test crate whose name is the same as the name of crate being
+  tested, with a `-gtest` suffix.
+- Add to the test crate a Rust file, a C++ file containing GTest `TEST()`
+  functions that use FFI to call into the Rust file, a `Cargo.toml` file that
+  references the Rust file, and a `moz.build` file that references the C++
+  file.
+- Add an entry to the `[dependencies]` section in
+  [toolkit/library/gtest/rust/Cargo.toml](https://searchfox.org/mozilla-central/source/toolkit/library/gtest/rust/Cargo.toml).
+- Add an `extern crate` entry to
+  [toolkit/library/gtest/rust/lib.rs](https://searchfox.org/mozilla-central/source/toolkit/library/gtest/rust/lib.rs).
+
+See
+[xpcom/rust/gtest/nsstring/](https://searchfox.org/mozilla-central/source/xpcom/rust/gtest/nsstring)
+for a simple example. (Note that the `moz.build` file is in the parent
+directory for that crate.)
+
+A Rust GTest can be run like any other GTest via `./mach gtest`, using the C++
+`TEST()` functions as the starting point.
+
+Unlike Rust tests, GTests can be used when linking against Gecko symbols is required.
+
+## Testing third-party crates
+
+In general we don't run tests for third-party crates. The assumption is that
+these crates are sufficiently well-tested elsewhere.
+
+## Debugging Rust code
+
+In theory, Rust code is debuggable much like C++ code, using standard tools
+like `gdb`, `rr`, and the Microsoft Visual Studio Debugger. In practice, the
+experience can be worse, because shortcomings such as the following can occur.
+- Inability to print local variables, even in non-optimized builds.
+- Inability to call generic functions.
+- Missing line numbers and stack frames.
+- Printing of basic types such as `Option` and `Vec` is sometimes sub-optimal.
+  If you see a warning "Missing auto-load script at offset 0 in section
+  `.debug_gdb_scripts`" when starting `gdb`, the `rust-gdb` wrapper may give
+  better results.
+
+## Logging from Rust code
+
+### Rust logging
+
+The `RUST_LOG` environment variable (from the `env_logger` crate) can be used
+to enable logging to stderr from Rust code in Firefox. The logging macros from
+the `log` crate can be used. In order of importance, they are: `error!`,
+`warn!`, `info!`, `debug!`, `trace!`.
+
+For example, to show all log messages of `info` level or higher, run:
+```
+RUST_LOG=info firefox
+```
+Module-level logging can also be specified, see the [documentation] for the
+`env_logger` crate for details.
+
+To restrict logging to child processes, use `RUST_LOG_CHILD` instead of
+`RUST_LOG`.
+
+[documentation]: https://docs.rs/env_logger/
+
+### Gecko logging
+
+Rust logging can also be forwarded to the [Gecko logger] for capture via
+`MOZ_LOG` and `MOZ_LOG_FILE`.
+
+[Gecko logger]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Gecko_Logging
+
+- When parsing modules from `MOZ_LOG`, modules containing `::` are considered
+  to be Rust modules. To log everything in a top-level module like
+  `neqo_transport`, specify it as `neqo_transport::*`. For example:
+```
+MOZ_LOG=timestamp,sync,nsHostResolver:5,neqo_transport::*:5,proxy:5 firefox
+```
+- When logging from a submodule the `::*` is allowed but isn't necessary.
+  So these two lines are equivalent:
+```
+MOZ_LOG=timestamp,sync,neqo_transport::recovery:5 firefox
+MOZ_LOG=timestamp,sync,neqo_transport::recovery::*:5 firefox
+```
+- `debug!` and `trace!` logs will not appear in non-debug builds. This is due
+  to our use of the `release_max_level_info` feature in the `log` crate.
+
+- When using both `MOZ_LOG` and `RUST_LOG`, modules that are specified in
+  `MOZ_LOG` will not appear in `RUST_LOG`.

moz.build

@@ -198,6 +198,8 @@ SPHINX_TREES['contributing'] = 'docs/contributing'
 
 SPHINX_TREES['code-quality'] = 'docs/code-quality'
 
+SPHINX_TREES['testing-rust-code'] = 'docs/testing-rust-code'
+
 SPHINX_TREES['bug-mgmt'] = 'docs/bug-mgmt'
 
 SPHINX_TREES['setup'] = 'docs/setup'