Bug 1560755 - Merge Android PGO profile data in run task; r=firefox-build-system-reviewers,chmanchester
authorMike Shal <mshal@mozilla.com>
Tue, 02 Jul 2019 21:51:23 +0000
changeset 543862 5a867f13f12e246e386a7f5050ce1d80e958c5d9
parent 543861 a6fa0472457f8406f3cac75c777cfeb5770913e2
child 543863 d066f9f4b7bcf5d3b692ca5a198406e1ff358076
push id2131
push userffxbld-merge
push dateMon, 26 Aug 2019 18:30:20 +0000
treeherdermozilla-release@b19ffb3ca153 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersfirefox-build-system-reviewers, chmanchester
bugs1560755
milestone69.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1560755 - Merge Android PGO profile data in run task; r=firefox-build-system-reviewers,chmanchester Android profile runs don't always fully write out the profile data. In this case, the corrupted profile data is successfully uploaded, but future profile-use PGO builds try to merge the data and fail. Retrying the profile-use builds doesn't help, since they all pull from the same job that published the corrupt data. We can detect this in the run task by using llvm-profdata merge, and if the merge fails the task can automatically be retried. Note that the data gets redundantly merged in the profile-use build, but it may not be possible to run the merge in the run task on all platforms (eg: OSX), so we have to keep the merge there as well. Differential Revision: https://phabricator.services.mozilla.com/D36294
taskcluster/ci/generate-profile/kind.yml
testing/mozharness/scripts/android_emulator_pgo.py
--- a/taskcluster/ci/generate-profile/kind.yml
+++ b/taskcluster/ci/generate-profile/kind.yml
@@ -1,21 +1,23 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 ---
 loader: taskgraph.loader.transform:loader
 
 kind-dependencies:
+    - toolchain
     - instrumented-build
 
 transforms:
     - taskgraph.transforms.build_attrs:transforms
     - taskgraph.transforms.release_deps:transforms
     - taskgraph.transforms.run_pgo_profile:transforms
+    - taskgraph.transforms.use_toolchains:transforms
     - taskgraph.transforms.job:transforms
     - taskgraph.transforms.task:transforms
 
 
 job-defaults:
     treeherder:
         symbol: Bpgo(run)
         kind: test
@@ -120,16 +122,18 @@ jobs:
             job-script: taskcluster/scripts/tester/test-linux.sh
             script: android_emulator_pgo.py
             tooltool-downloads: internal
             options: [installer-path=/builds/worker/fetches/target.apk]
             config:
                 - android/android_common.py
                 - android/androidarm_4_3.py
                 - android/android_pgo.py
+        toolchains:
+            - linux64-clang
 
     win32-shippable/opt:
         description: "Win32 Profile Generation"
         shipping-phase: build
         shipping-product: firefox
         index:
             product: firefox
             job-name: win32-profile
--- a/testing/mozharness/scripts/android_emulator_pgo.py
+++ b/testing/mozharness/scripts/android_emulator_pgo.py
@@ -272,23 +272,39 @@ class AndroidProfileRun(TestingMixin, Ba
                     time.sleep(2)
             else:
                 raise Exception("Unable to pull default.profraw")
             adbdevice.pull(jarlog, '/builds/worker/workspace/en-US.log')
         except ADBTimeoutError:
             self.fatal('INFRA-ERROR: Failed with an ADBTimeoutError',
                        EXIT_STATUS_DICT[TBPL_RETRY])
 
+        # We normally merge as part of a GENERATED_FILES step in the profile-use
+        # build, but Android runs sometimes result in a truncated profile. We do
+        # a merge here to make sure the data isn't corrupt so we can retry the
+        # 'run' task if necessary.
+        merge_cmd = [
+            '/builds/worker/workspace/build/clang/bin/llvm-profdata',
+            'merge',
+            '/builds/worker/workspace/default.profraw',
+            '-o',
+            '/builds/worker/workspace/merged.profraw',
+        ]
+        rc = subprocess.call(merge_cmd)
+        if rc != 0:
+            self.fatal('INFRA-ERROR: Failed to merge profile data. Corrupt profile?',
+                       EXIT_STATUS_DICT[TBPL_RETRY])
+
         # tarfile doesn't support xz in this version of Python
         tar_cmd = [
             'tar',
             '-acvf',
             '/builds/worker/artifacts/profdata.tar.xz',
             '-C', '/builds/worker/workspace',
-            'default.profraw',
+            'merged.profraw',
             'en-US.log',
         ]
         subprocess.check_call(tar_cmd)
 
         httpd.stop()
 
 
 if __name__ == '__main__':