bug 1606834: remote: document fdescribe(), fit(), and Mocha flags r=remote-protocol-reviewers,whimboo
authorAndreas Tolfsen <ato@sny.no>
Fri, 03 Jan 2020 16:11:19 +0000
changeset 508859 6c6336d7656d279ae3fbad36f035594a57bf1da7
parent 508858 88e8e85320ed2036b46eab8039344fe1939f0b5a
child 508860 346f1f020530528a26981d21e0d8d8e019124d63
push id36986
push usernerli@mozilla.com
push dateMon, 06 Jan 2020 21:54:03 +0000
treeherdermozilla-central@e6427fac5ee8 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersremote-protocol-reviewers, whimboo
bugs1606834
milestone73.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
bug 1606834: remote: document fdescribe(), fit(), and Mocha flags r=remote-protocol-reviewers,whimboo We were lacking documentation on how to run inidividual Puppeteer tests. It turns out this is not possible yet to do on a per-file basis, like other test harnesses. Instead, suggest use of fdescribe()/fit() and recommend some useful Mocha flags. Differential Revision: https://phabricator.services.mozilla.com/D58618
remote/doc/Testing.md
--- a/remote/doc/Testing.md
+++ b/remote/doc/Testing.md
@@ -103,23 +103,66 @@ original `add_task()`.
 [asynchronous tests]: https://developer.mozilla.org/en-US/docs/Mozilla/Browser_chrome_tests#Test_functions
 [CDP client]: https://github.com/cyrus-and/chrome-remote-interface
 
 
 Puppeteer tests
 ---------------
 
 In addition to our own Firefox-specific tests, we run the upstream
-[Puppeteer test suite] against our implementation to track progress
+[Puppeteer test suite] against our implementation to [track progress]
 towards achieving full [Puppeteer support] in Firefox.
 
 These tests are vendored under _remote/test/puppeteer/_ and are
 run locally like this:
 
 	% ./mach test remote/test/puppeteer/test
 
 On try they appear under the `remote(pup)` symbol, but because they’re
 a Tier-3 class test job they’re not automatically scheduled.
 To schedule the tests, look for `source-test-remote-puppeteer` in
 `./mach try fuzzy`.
 
+The tests are written in the behaviour-driven testing framework
+[Mocha], which doesn’t support selecting tests by file path like
+other harnesses.  It does however come with a myriad of flags to
+narrow down the selection a bit: some of the most useful ones include
+[`--grep`], [`--ignore`], and [`--file`].
+
+Perhaps the most useful trick is to rename the `describe()` or
+`it()` functions to `fdescribe()` and `fit()`, respectively, in
+order to run a specific subset of tests.  This does however force
+you to edit the test files manually.
+
+A real-world example:
+
+```diff
+diff --git a/remote/test/puppeteer/test/frame.spec.js b/remote/test/puppeteer/test/frame.spec.js
+index 58e57934a7b8..0531d49d7a12 100644
+--- a/remote/test/puppeteer/test/frame.spec.js
++++ b/remote/test/puppeteer/test/frame.spec.js
+@@ -48,7 +48,7 @@ module.exports.addTests = function({testRunner, expect}) {
+     });
+   });
+
+-  describe('Frame.evaluateHandle', function() {
++  fdescribe('Frame.evaluateHandle', function() {
+     it('should work', async({page, server}) => {
+       await page.goto(server.EMPTY_PAGE);
+       const mainFrame = page.mainFrame();
+@@ -58,7 +58,7 @@ module.exports.addTests = function({testRunner, expect}) {
+   });
+
+   describe('Frame.evaluate', function() {
+-    it('should throw for detached frames', async({page, server}) => {
++    fit('should throw for detached frames', async({page, server}) => {
+       const frame1 = await utils.attachFrame(page, 'frame1', server.EMPTY_PAGE);
+       await utils.detachFrame(page, 'frame1');
+       let error = null;
+```
+
 [Puppeteer test suite]: https://github.com/GoogleChrome/puppeteer/tree/master/test
+[track progress]: https://docs.google.com/spreadsheets/d/1GZ4yO2-NGD6kbsT7aMlUPUpUqTaASpqNxJGKhOQ-_BM/edit#gid=0
 [Puppeteer support]: https://bugzilla.mozilla.org/show_bug.cgi?id=puppeteer
+[Mocha]: https://mochajs.org/
+[`--grep`]: https://mochajs.org/#-grep-regexp-g-regexp
+[`--ignore`]: https://mochajs.org/#-ignore-filedirectoryglob
+[`--file`]: https://mochajs.org/#-file-filedirectoryglob