Bug 1338534 - [mozlog] Indicate unexpected pass with the correct status. r=ahal
authorDave Hunt <dhunt@mozilla.com>
Tue, 14 Feb 2017 23:09:28 +0000
changeset 343302 b854735013bbbd22e800e4a6c4031475be7ff110
parent 343301 501c0a140c82d87d863eb02f5a4335e04cef7670
child 343303 9b09b5b2f7292b89dfd204f15dae6f8ebef94196
push id31376
push userkwierso@gmail.com
push dateFri, 17 Feb 2017 01:09:40 +0000
treeherdermozilla-central@6cefe01ca774 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersahal
bugs1338534
milestone54.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1338534 - [mozlog] Indicate unexpected pass with the correct status. r=ahal Unexpected passes can cause failures in pytest when the expected failures are set to strict. When we're logging these we want to indicate that the correct outcome. MozReview-Commit-ID: 8VoR6CTrynY
testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
--- a/testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
+++ b/testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
@@ -77,16 +77,20 @@ class MozLog(object):
         if report.failed:
             status = 'FAIL' if report.when == 'call' else 'ERROR'
         if report.skipped:
             status = 'SKIP' if not hasattr(report, 'wasxfail') else 'FAIL'
         if report.longrepr is not None:
             if isinstance(report.longrepr, basestring):
                 # When using pytest-xdist, longrepr is serialised as a str
                 message = stack = report.longrepr
+                if report.longrepr.startswith('[XPASS(strict)]'):
+                    # Strict expected failures have an outcome of failed when
+                    # they unexpectedly pass.
+                    expected, status = ('FAIL', 'PASS')
             else:
                 try:
                     # For failures, longrepr is a ReprExceptionInfo
                     crash = report.longrepr.reprcrash
                     message = "{0} (line {1})".format(crash.message, crash.lineno)
                     stack = report.longrepr.reprtraceback
                 except AttributeError:
                     # For skips, longrepr is a tuple of (file, lineno, reason)