Bug 1370421 - Put base-toolchains builds into own perfherder bucket; r=mshal
authorGregory Szorc <gps@mozilla.com>
Tue, 06 Jun 2017 16:23:14 -0700
changeset 413342 378499da20301b4d3952a5068f4a1422cad45ef4
parent 413341 858813bdc9b43c07d48d55fc85801c0b0a65144c
child 413343 d2289ec1f6d1a87f40f02b8b933dddc1d3ba5c1a
push id1490
push usermtabara@mozilla.com
push dateMon, 31 Jul 2017 14:08:16 +0000
treeherdermozilla-release@70e32e6bf15e [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersmshal
bugs1370421
milestone55.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1370421 - Put base-toolchains builds into own perfherder bucket; r=mshal Perfherder metrics are by default put in a bucket according to their platform name. e.g. linux64/opt. Similar jobs emitting otherwise identical metric names must differentiate the metrics by inserting extra metadata. Before this commit, we defined extra metadata in mozharness sub-configs, as appropriate. As long as the sub-config remembers to define the metadata, we're good. However, some tasks don't use mozharness sub-configs. Instead, they create variations in the taskgraph. The base toolchains builds are an example. These share the same platform name (e.g. linux64/opt) and the base mozharness config. Therefore there's no way to distinguish these builds as being "base toolchains" from mozharness. This commit teaches mozharness to look for an environment variable defining extra perfherder keys to use to bucket results. We define this variable in taskgraph for relevant tasks. MozReview-Commit-ID: 8oGHGyx6Zn5
taskcluster/ci/build/linux.yml
testing/mozharness/mozharness/mozilla/building/buildbase.py
--- a/taskcluster/ci/build/linux.yml
+++ b/taskcluster/ci/build/linux.yml
@@ -102,16 +102,17 @@ linux64-base-toolchains/opt:
         job-name: linux64-base-toolchains-opt
     treeherder:
         platform: linux64/opt
         symbol: tc(Bb)
     worker-type: aws-provisioner-v1/gecko-{level}-b-linux
     worker:
         max-run-time: 36000
         env:
+            PERFHERDER_EXTRA_OPTIONS: base-toolchains
             TOOLTOOL_MANIFEST: browser/config/tooltool-manifests/linux64/base-toolchains.manifest
     run:
         using: mozharness
         actions: [get-secrets build check-test update]
         config:
             - builds/releng_base_linux_64_builds.py
             - balrog/production.py
         script: "mozharness/scripts/fx_desktop_build.py"
@@ -126,16 +127,17 @@ linux64-base-toolchains/debug:
         job-name: linux64-base-toolchains-debug
     treeherder:
         platform: linux64/debug
         symbol: tc(Bb)
     worker-type: aws-provisioner-v1/gecko-{level}-b-linux
     worker:
         max-run-time: 36000
         env:
+            PERFHERDER_EXTRA_OPTIONS: base-toolchains
             TOOLTOOL_MANIFEST: browser/config/tooltool-manifests/linux64/base-toolchains.manifest
     run:
         using: mozharness
         actions: [get-secrets build check-test update]
         config:
             - builds/releng_base_linux_64_builds.py
             - balrog/production.py
         script: "mozharness/scripts/fx_desktop_build.py"
--- a/testing/mozharness/mozharness/mozilla/building/buildbase.py
+++ b/testing/mozharness/mozharness/mozilla/building/buildbase.py
@@ -2101,16 +2101,21 @@ or run without that action (ie: --no-{ac
         perfherder_data['suites'].extend(self._load_sccache_stats())
 
         # Ensure all extra options for this configuration are present.
         for opt in self.config.get('perfherder_extra_options', []):
             for suite in perfherder_data['suites']:
                 if opt not in suite.get('extraOptions', []):
                     suite.setdefault('extraOptions', []).append(opt)
 
+        for opt in os.environ.get('PERFHERDER_EXTRA_OPTIONS', '').split():
+            for suite in perfherder_data['suites']:
+                if opt not in suite.get('extraOptions', []):
+                    suite.setdefault('extraOptions', []).append(opt)
+
         if self.query_is_nightly():
             for suite in perfherder_data['suites']:
                 suite.setdefault('extraOptions', []).insert(0, 'nightly')
 
         if perfherder_data["suites"]:
             self.info('PERFHERDER_DATA: %s' % json.dumps(perfherder_data))
 
     def sendchange(self):