Bug 1560755 - Merge Android PGO profile data in run task; r=firefox-build-system-reviewers,chmanchester
authorMike Shal <mshal@mozilla.com>
Tue, 02 Jul 2019 21:51:23 +0000
changeset 543862 5a867f13f12e246e386a7f5050ce1d80e958c5d9
parent 543861 a6fa0472457f8406f3cac75c777cfeb5770913e2
child 543863 d066f9f4b7bcf5d3b692ca5a198406e1ff358076
push id2131
push userffxbld-merge
push dateMon, 26 Aug 2019 18:30:20 +0000
treeherdermozilla-release@b19ffb3ca153 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersfirefox-build-system-reviewers, chmanchester
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1560755 - Merge Android PGO profile data in run task; r=firefox-build-system-reviewers,chmanchester Android profile runs don't always fully write out the profile data. In this case, the corrupted profile data is successfully uploaded, but future profile-use PGO builds try to merge the data and fail. Retrying the profile-use builds doesn't help, since they all pull from the same job that published the corrupt data. We can detect this in the run task by using llvm-profdata merge, and if the merge fails the task can automatically be retried. Note that the data gets redundantly merged in the profile-use build, but it may not be possible to run the merge in the run task on all platforms (eg: OSX), so we have to keep the merge there as well. Differential Revision: https://phabricator.services.mozilla.com/D36294
--- a/taskcluster/ci/generate-profile/kind.yml
+++ b/taskcluster/ci/generate-profile/kind.yml
@@ -1,21 +1,23 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 loader: taskgraph.loader.transform:loader
+    - toolchain
     - instrumented-build
     - taskgraph.transforms.build_attrs:transforms
     - taskgraph.transforms.release_deps:transforms
     - taskgraph.transforms.run_pgo_profile:transforms
+    - taskgraph.transforms.use_toolchains:transforms
     - taskgraph.transforms.job:transforms
     - taskgraph.transforms.task:transforms
         symbol: Bpgo(run)
         kind: test
@@ -120,16 +122,18 @@ jobs:
             job-script: taskcluster/scripts/tester/test-linux.sh
             script: android_emulator_pgo.py
             tooltool-downloads: internal
             options: [installer-path=/builds/worker/fetches/target.apk]
                 - android/android_common.py
                 - android/androidarm_4_3.py
                 - android/android_pgo.py
+        toolchains:
+            - linux64-clang
         description: "Win32 Profile Generation"
         shipping-phase: build
         shipping-product: firefox
             product: firefox
             job-name: win32-profile
--- a/testing/mozharness/scripts/android_emulator_pgo.py
+++ b/testing/mozharness/scripts/android_emulator_pgo.py
@@ -272,23 +272,39 @@ class AndroidProfileRun(TestingMixin, Ba
                 raise Exception("Unable to pull default.profraw")
             adbdevice.pull(jarlog, '/builds/worker/workspace/en-US.log')
         except ADBTimeoutError:
             self.fatal('INFRA-ERROR: Failed with an ADBTimeoutError',
+        # We normally merge as part of a GENERATED_FILES step in the profile-use
+        # build, but Android runs sometimes result in a truncated profile. We do
+        # a merge here to make sure the data isn't corrupt so we can retry the
+        # 'run' task if necessary.
+        merge_cmd = [
+            '/builds/worker/workspace/build/clang/bin/llvm-profdata',
+            'merge',
+            '/builds/worker/workspace/default.profraw',
+            '-o',
+            '/builds/worker/workspace/merged.profraw',
+        ]
+        rc = subprocess.call(merge_cmd)
+        if rc != 0:
+            self.fatal('INFRA-ERROR: Failed to merge profile data. Corrupt profile?',
+                       EXIT_STATUS_DICT[TBPL_RETRY])
         # tarfile doesn't support xz in this version of Python
         tar_cmd = [
             '-C', '/builds/worker/workspace',
-            'default.profraw',
+            'merged.profraw',
 if __name__ == '__main__':