Bug 1519862 - Give mozharness output to indicate why a job failed, r=ahal
authorJames Graham <james@hoppipolla.co.uk>
Thu, 10 Jan 2019 11:06:19 +0000
changeset 515070 765ec7f392431213b37a7faa1e9ffbc4ccbba85b
parent 515069 f7f763f7c5b4771be1809898a9455ceec1d88515
child 515071 add585284f4ed5a40485f5301e7ed13e6c1c354f
push id1953
push userffxbld-merge
push dateMon, 11 Mar 2019 12:10:20 +0000
treeherdermozilla-release@9c35dcbaa899 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersahal
bugs1519862
milestone66.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1519862 - Give mozharness output to indicate why a job failed, r=ahal Sometimes it's unclear which of the errors are leading to a failure. For example wpt sometimes has PROCESS-CRASH lines that don't contribute to a failure. To make this easier to understand record the reason for the failure in the log so it appears in the summaries. Differential Revision: https://phabricator.services.mozilla.com/D16835
testing/mozharness/mozharness/mozilla/structuredlog.py
--- a/testing/mozharness/mozharness/mozilla/structuredlog.py
+++ b/testing/mozharness/mozharness/mozilla/structuredlog.py
@@ -174,23 +174,29 @@ class StructuredOutputParser(OutputParse
         else:
             joined_summary = summary
 
         fail_pair = TBPL_WARNING, WARNING
         error_pair = TBPL_FAILURE, ERROR
 
         # These are warning/orange statuses.
         failure_conditions = [
-            sum(summary.unexpected_statuses.values()) > 0,
-            summary.action_counts.get('crash', 0) > summary.expected_statuses.get('CRASH', 0),
-            summary.action_counts.get('valgrind_error', 0) > 0
+            (sum(summary.unexpected_statuses.values()), 0, "statuses"),
+            (summary.action_counts.get('crash', 0),
+             summary.expected_statuses.get('CRASH', 0), "crashes"),
+            (summary.action_counts.get('valgrind_error', 0), 0,
+             "valgrind errors")
         ]
-        for condition in failure_conditions:
-            if condition:
+        for value, limit, type_name in failure_conditions:
+            if value > limit:
                 self.update_levels(*fail_pair)
+                msg = "Got %d unexpected %s" % (value, type_name)
+                if limit != 0:
+                    msg += " expected at most %d" % (limit)
+                self.error(msg)
 
         # These are error/red statuses. A message is output here every time something
         # wouldn't otherwise be highlighted in the UI.
         required_actions = {
             'suite_end': 'No suite end message was emitted by this harness.',
             'test_end': 'No checks run.',
         }
         for action, diagnostic_message in required_actions.iteritems():