Bug 1595180 - Report assertions in Mochitests with fail-if annotations more sensibly. r=ahal
authorKris Maglione <maglione.k@gmail.com>
Tue, 12 Nov 2019 21:41:14 +0000
changeset 501667 b7dc2082925ee821199b92c1d48edfd0da0b6aef
parent 501666 7019a4a1fe39964f90b14158e84d05d727d888bd
child 501668 bf5bee09a2fccd063b071395355ecd48856d6c69
push id114172
push userdluca@mozilla.com
push dateTue, 19 Nov 2019 11:31:10 +0000
treeherdermozilla-inbound@b5c5ba07d3db [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersahal
bugs1595180
milestone72.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1595180 - Report assertions in Mochitests with fail-if annotations more sensibly. r=ahal Currently, when a Mochitest has a fail-if annotation, any passing assertions are reported as EXPECTED-FAIL, which is extremely confusing, and makes it difficult to understand which parts of the test are still failing without removing the annotation. What we really want is for passing assertions to still be reported as passing, and for only actually failing assertions to reported as EXPECTED-FAIL, which this patch does. Differential Revision: https://phabricator.services.mozilla.com/D52407
testing/mochitest/tests/SimpleTest/SimpleTest.js
--- a/testing/mochitest/tests/SimpleTest/SimpleTest.js
+++ b/testing/mochitest/tests/SimpleTest/SimpleTest.js
@@ -280,18 +280,18 @@ SimpleTest.ok = function (condition, nam
 
 SimpleTest.record = function (condition, name, diag, stack) {
     var test = {'result': !!condition, 'name': name, 'diag': diag};
     if (SimpleTest.expected == 'fail') {
       if (!test.result) {
         SimpleTest.num_failed++;
         test.result = !test.result;
       }
-      var successInfo = {status:"FAIL", expected:"FAIL", message:"TEST-KNOWN-FAIL"};
-      var failureInfo = {status:"PASS", expected:"FAIL", message:"TEST-UNEXPECTED-PASS"};
+      var successInfo = {status:"PASS", expected:"PASS", message:"TEST-PASS"};
+      var failureInfo = {status:"FAIL", expected:"FAIL", message:"TEST-KNOWN-FAIL"};
     } else if (!test.result && usesFailurePatterns()) {
       if (recordIfMatchesFailurePattern(name, diag)) {
         test.result = true;
         // Add a mark for unexpected failures suppressed by failure pattern.
         name = '[suppressed] ' + name;
       }
       var successInfo = {status:"FAIL", expected:"FAIL", message:"TEST-KNOWN-FAIL"};
       var failureInfo = {status:"FAIL", expected:"PASS", message:"TEST-UNEXPECTED-FAIL"};