Bug 1647987 - Create Rust testing docs. r=froydnj.
authorNicholas Nethercote <nnethercote@mozilla.com>
Fri, 26 Jun 2020 01:04:40 +0000
changeset 537533 b1a22f246546b5c5e7a9f55df062f755cd859348
parent 537532 979f641a6bef7cb476dae782245ce7898d634c24
child 537534 5e0244588a9158f6570b58bde8fc90abc14dedfd
push id37544
push userdluca@mozilla.com
push dateFri, 26 Jun 2020 09:45:40 +0000
treeherdermozilla-central@9b981fcfb6ac [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersfroydnj
bugs1647987
milestone79.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1647987 - Create Rust testing docs. r=froydnj. Some of the testing info is from the Oxidation wiki, and the logging info is largely from a dev-platform email by Valentin. The other parts I wrote from scratch. The commit also makes some small improvements to the Rust build docs. Differential Revision: https://phabricator.services.mozilla.com/D81017
build/docs/rust.rst
docs/config.yml
docs/testing-rust-code/index.md
moz.build
--- a/build/docs/rust.rst
+++ b/build/docs/rust.rst
@@ -8,27 +8,27 @@ The build system has support for buildin
 It is straightforward to take an existing Rust crate and integrate it into
 Firefox.
 
 Linking Rust crates into libxul
 ===============================
 
 Rust crates that you want to link into libxul should be listed in the
 ``dependencies`` section of
-`toolkit/library/rust/shared/Cargo.toml <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml>`_.
+`toolkit/library/rust/shared/Cargo.toml <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/Cargo.toml>`_.
 After adding your crate, execute ``cargo update -p gkrust-shared``
 to update the ``Cargo.lock`` file. You'll also need to add an ``extern crate``
 reference to
-`toolkit/library/rust/shared/lib.rs <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_.
+`toolkit/library/rust/shared/lib.rs <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_.
 This ensures that the Rust code will be linked properly into libxul as well
 as the copy of libxul used for gtests.
 
 By default, all Cargo packages in the mozilla-central repository are part of
 the same
-`workspace <https://dxr.mozilla.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_
+`workspace <https://searchfox.org/mozilla-central/source/toolkit/library/rust/shared/lib.rs>`_
 and will share the ``Cargo.lock`` file and ``target`` directory in the root of
 the repository.  You can change this behavior by adding a path to the
 ``exclude`` list in the top-level ``Cargo.toml`` file.  You may want to do
 this if your package's development workflow includes dev-dependencies that
 aren't needed by general Firefox developers or test infrastructure.
 
 The actual build mechanism is as follows. The build system generates a special
 'Rust unified library' crate, compiles that to a static library
@@ -44,17 +44,17 @@ Linking Rust crates into something else
 To link Rust code into libraries other than libxul, create a directory with a
 ``Cargo.toml`` file for your crate, and a ``moz.build`` file that contains:
 
 .. code-block:: python
 
     RustLibrary('crate_name')
 
 where ``crate_name`` matches the name from the ``[package]`` section of your
-``Cargo.toml``. You can refer to `the moz.build file <https://dxr.mozilla.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/moz.build#7>`_ and `the Cargo.toml file <https://dxr.mozilla.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/Cargo.toml>`_ that are used for libxul.
+``Cargo.toml``. You can refer to `the moz.build file <https://searchfox.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/moz.build#7>`_ and `the Cargo.toml file <https://searchfox.org/mozilla-central/rev/3f4c3a3cabaf94958834d3a8935adfb4a887942d/toolkit/library/rust/Cargo.toml>`_ that are used for libxul.
 
 You can then add ``USE_LIBS += ['crate_name']`` to the ``moz.build`` file
 that defines the binary as you would with any other library in the tree.
 
 .. important::
 
     You cannot link a Rust crate into an intermediate library that will be
     eventually linked into libxul. The build system enforces that only a single
@@ -114,16 +114,20 @@ vendor the dependencies into that direct
 as on other platforms.)
 
 When it comes to checking the suitability of third-party code for inclusion
 into mozilla-central, keep the following in mind.
 
 - ``mach vendor rust`` will check that the licenses of all crates are suitable.
 - You should review the crate code to some degree to check that it looks
   reasonable (especially for unsafe code) and that it has reasonable tests.
+- Third-party crate tests aren't run, which means that large test fixtures will
+  bloat mozilla-central. Consider working with upstream to mark those test
+  fixtures with ``[package] exclude = ...`` as described
+  `here <https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields>`_.
 - Other than that, there is no formal sign-off procedure, but one may be added
   in the future.
 
 Note that all dependencies will be vendored, even ones that aren't used due to
 disabled features. It's possible that multiple versions of a crate will end up
 vendored into mozilla-central.
 
 Patching third-party crates
@@ -143,8 +147,14 @@ example
 Next, run ``cargo update -p $CRATE_NAME --precise $VERSION``, where
 ``$CRATE_NAME`` is the name of the patched crate, and ``$VERSION`` is its
 version number. This will update the ``Cargo.lock`` file.
 
 Then, make the local changes to the crate.
 
 Finally, make sure you don't accidentally land the changes to the crate or the
 ``Cargo.lock`` file.
+
+For an example of a more complex workflow involving a third-party crate, see
+`mp4parse-rust/README.md <https://searchfox.org/mozilla-central/source/media/mp4parse-rust/README.md>`_.
+It describes the workflow for a crate that is hosted on GitHub, and for which
+changes are made via GitHub pull requests, but all pull requests must also be
+tested within mozilla-central before being merged.
--- a/docs/config.yml
+++ b/docs/config.yml
@@ -34,16 +34,17 @@ categories:
         - testing/marionette
         - testing/geckodriver
         - web-platform
         - tools/fuzzing
         - tools/sanitizer
         - testing/perfdocs
         - testing/perftest
         - tools/code-coverage
+        - testing-rust-code
     l10n_doc:
         - intl
         - l10n
     python_doc:
         - mozbase
         - python
     fennec_doc:
         - mobile/android
new file mode 100644
--- /dev/null
+++ b/docs/testing-rust-code/index.md
@@ -0,0 +1,117 @@
+# Testing & Debugging Rust Code
+
+## Testing Mozilla crates
+
+Rust code will naturally be tested as part of system tests such as Mochitests.
+This section describes the two methods for unit testing of individual Rust
+crates. Which method should be used depends on the circumstances.
+
+### Rust tests
+
+If a Mozilla crate has "normal" Rust tests (i.e. `#[test]` functions that run
+with `cargo test`), you can add the crate's name to `RUST_TESTS` in
+[toolkit/library/rust/moz.build](https://searchfox.org/mozilla-central/source/toolkit/library/rust/moz.build).
+(Cargo features can be activated for Rust tests by adding them to
+`RUST_TEST_FEATURES` in the same file.)
+
+Rust tests are run with `./mach rusttests`. They run on automation in a couple
+of `rusttests` jobs, but not on all platforms.
+
+Rust tests have one major restriction: they cannot link against Gecko symbols.
+Therefore, Rust tests cannot be used for crates that use Gecko crates like
+`nsstring` and `xpcom`.
+
+It's also possible to use `RUST_TESTS` in a different `moz.build` file. See
+`testing/geckodriver/moz.build` and the [geckodriver testing docs] for an
+example.
+
+[geckodriver testing docs]: ../testing/geckodriver/Testing.html
+
+### GTests
+
+Another way to unit test a Mozilla crate is by writing a GTest that uses FFI to
+call into Rust code. This requires the following steps.
+- Create a new test crate whose name is the same as the name of crate being
+  tested, with a `-gtest` suffix.
+- Add to the test crate a Rust file, a C++ file containing GTest `TEST()`
+  functions that use FFI to call into the Rust file, a `Cargo.toml` file that
+  references the Rust file, and a `moz.build` file that references the C++
+  file.
+- Add an entry to the `[dependencies]` section in
+  [toolkit/library/gtest/rust/Cargo.toml](https://searchfox.org/mozilla-central/source/toolkit/library/gtest/rust/Cargo.toml).
+- Add an `extern crate` entry to
+  [toolkit/library/gtest/rust/lib.rs](https://searchfox.org/mozilla-central/source/toolkit/library/gtest/rust/lib.rs).
+
+See
+[xpcom/rust/gtest/nsstring/](https://searchfox.org/mozilla-central/source/xpcom/rust/gtest/nsstring)
+for a simple example. (Note that the `moz.build` file is in the parent
+directory for that crate.)
+
+A Rust GTest can be run like any other GTest via `./mach gtest`, using the C++
+`TEST()` functions as the starting point.
+
+Unlike Rust tests, GTests can be used when linking against Gecko symbols is required.
+
+## Testing third-party crates
+
+In general we don't run tests for third-party crates. The assumption is that
+these crates are sufficiently well-tested elsewhere.
+
+## Debugging Rust code
+
+In theory, Rust code is debuggable much like C++ code, using standard tools
+like `gdb`, `rr`, and the Microsoft Visual Studio Debugger. In practice, the
+experience can be worse, because shortcomings such as the following can occur.
+- Inability to print local variables, even in non-optimized builds.
+- Inability to call generic functions.
+- Missing line numbers and stack frames.
+- Printing of basic types such as `Option` and `Vec` is sometimes sub-optimal.
+  If you see a warning "Missing auto-load script at offset 0 in section
+  `.debug_gdb_scripts`" when starting `gdb`, the `rust-gdb` wrapper may give
+  better results.
+
+## Logging from Rust code
+
+### Rust logging
+
+The `RUST_LOG` environment variable (from the `env_logger` crate) can be used
+to enable logging to stderr from Rust code in Firefox. The logging macros from
+the `log` crate can be used. In order of importance, they are: `error!`,
+`warn!`, `info!`, `debug!`, `trace!`.
+
+For example, to show all log messages of `info` level or higher, run:
+```
+RUST_LOG=info firefox
+```
+Module-level logging can also be specified, see the [documentation] for the
+`env_logger` crate for details.
+
+To restrict logging to child processes, use `RUST_LOG_CHILD` instead of
+`RUST_LOG`.
+
+[documentation]: https://docs.rs/env_logger/
+
+### Gecko logging
+
+Rust logging can also be forwarded to the [Gecko logger] for capture via
+`MOZ_LOG` and `MOZ_LOG_FILE`.
+
+[Gecko logger]: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Gecko_Logging
+
+- When parsing modules from `MOZ_LOG`, modules containing `::` are considered
+  to be Rust modules. To log everything in a top-level module like
+  `neqo_transport`, specify it as `neqo_transport::*`. For example:
+```
+MOZ_LOG=timestamp,sync,nsHostResolver:5,neqo_transport::*:5,proxy:5 firefox
+```
+- When logging from a submodule the `::*` is allowed but isn't necessary.
+  So these two lines are equivalent:
+```
+MOZ_LOG=timestamp,sync,neqo_transport::recovery:5 firefox
+MOZ_LOG=timestamp,sync,neqo_transport::recovery::*:5 firefox
+```
+- `debug!` and `trace!` logs will not appear in non-debug builds. This is due
+  to our use of the `release_max_level_info` feature in the `log` crate.
+
+- When using both `MOZ_LOG` and `RUST_LOG`, modules that are specified in
+  `MOZ_LOG` will not appear in `RUST_LOG`.
--- a/moz.build
+++ b/moz.build
@@ -193,13 +193,15 @@ else:
 OBJDIR_PP_FILES['.cargo'] += ['.cargo/config.in']
 
 DEFINES['top_srcdir'] = TOPSRCDIR
 
 SPHINX_TREES['contributing'] = 'docs/contributing'
 
 SPHINX_TREES['code-quality'] = 'docs/code-quality'
 
+SPHINX_TREES['testing-rust-code'] = 'docs/testing-rust-code'
+
 SPHINX_TREES['bug-mgmt'] = 'docs/bug-mgmt'
 
 SPHINX_TREES['setup'] = 'docs/setup'
 
 include('build/templates.mozbuild')