Bug 1433923 - Use detailed failure message for wdspec test failures (pytest). r=jgraham a=test-only
authorHenrik Skupin <mail@hskupin.info>
Thu, 22 Mar 2018 13:16:33 +0100
changeset 462879 f66887206a10ef7d176226b693be208952bc3868
parent 462878 861a4eb681ccaf3f1c5b10dc2b4a44a13b7b73ea
child 462880 797a21d3bca62037bf58cb0a849eddbeb0202800
push id1683
push usersfraser@mozilla.com
push dateThu, 26 Apr 2018 16:43:40 +0000
treeherdermozilla-release@5af6cb21869d [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersjgraham, test-only
bugs1433923
milestone60.0
Bug 1433923 - Use detailed failure message for wdspec test failures (pytest). r=jgraham a=test-only When adding failures to the test report, the hook for pytest_runtest_logreport missed to extract the failure message from the stack, and as such passed in no message for the failure. MozReview-Commit-ID: CRhnnONMzbP
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/pytestrunner/runner.py
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/pytestrunner/runner.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/pytestrunner/runner.py
@@ -97,17 +97,29 @@ class SubtestResultRecorder(object):
                 self.record_fail(report)
         elif report.skipped:
             self.record_skip(report)
 
     def record_pass(self, report):
         self.record(report.nodeid, "PASS")
 
     def record_fail(self, report):
-        self.record(report.nodeid, "FAIL", stack=report.longrepr)
+        # pytest outputs the stacktrace followed by an error message prefixed
+        # with "E   ", e.g.
+        #
+        #        def test_example():
+        #  >         assert "fuu" in "foobar"
+        #  > E       AssertionError: assert 'fuu' in 'foobar'
+        message = ""
+        for line in report.longreprtext.splitlines():
+           if line.startswith("E   "):
+               message = line[1:].strip()
+               break
+
+        self.record(report.nodeid, "FAIL", message=message, stack=report.longrepr)
 
     def record_error(self, report):
         # error in setup/teardown
         if report.when != "call":
             message = "%s error" % report.when
         self.record(report.nodeid, "ERROR", message, report.longrepr)
 
     def record_skip(self, report):