Bug 1338534 - [mozlog] Indicate unexpected pass with the correct status. r=ahal
authorDave Hunt <dhunt@mozilla.com>
Tue, 14 Feb 2017 23:09:28 +0000
changeset 485679 b854735013bbbd22e800e4a6c4031475be7ff110
parent 485678 501c0a140c82d87d863eb02f5a4335e04cef7670
child 485680 9b09b5b2f7292b89dfd204f15dae6f8ebef94196
push id45811
push userbmo:markh@mozilla.com
push dateFri, 17 Feb 2017 01:50:29 +0000
reviewersahal
bugs1338534
milestone54.0a1
Bug 1338534 - [mozlog] Indicate unexpected pass with the correct status. r=ahal Unexpected passes can cause failures in pytest when the expected failures are set to strict. When we're logging these we want to indicate that the correct outcome. MozReview-Commit-ID: 8VoR6CTrynY
testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
--- a/testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
+++ b/testing/mozbase/mozlog/mozlog/pytest_mozlog/plugin.py
@@ -77,16 +77,20 @@ class MozLog(object):
         if report.failed:
             status = 'FAIL' if report.when == 'call' else 'ERROR'
         if report.skipped:
             status = 'SKIP' if not hasattr(report, 'wasxfail') else 'FAIL'
         if report.longrepr is not None:
             if isinstance(report.longrepr, basestring):
                 # When using pytest-xdist, longrepr is serialised as a str
                 message = stack = report.longrepr
+                if report.longrepr.startswith('[XPASS(strict)]'):
+                    # Strict expected failures have an outcome of failed when
+                    # they unexpectedly pass.
+                    expected, status = ('FAIL', 'PASS')
             else:
                 try:
                     # For failures, longrepr is a ReprExceptionInfo
                     crash = report.longrepr.reprcrash
                     message = "{0} (line {1})".format(crash.message, crash.lineno)
                     stack = report.longrepr.reprtraceback
                 except AttributeError:
                     # For skips, longrepr is a tuple of (file, lineno, reason)