Bug 1401725 - Use sparse checkouts for toolchain tasks; r=glandium
☠☠ backed out by e66f516aa655 ☠ ☠
authorGregory Szorc <gps@mozilla.com>
Thu, 21 Sep 2017 07:30:49 -0700
changeset 382197 af6c2544903e3281baf169b682daeef515f77ee3
parent 382196 76d2a93a4549a9f22b0dc6ef1095f4409d7cd267
child 382198 e66f516aa655169eacba03634c0c130a5f6d4e76
push id32551
push userkwierso@gmail.com
push dateThu, 21 Sep 2017 23:29:53 +0000
treeherdermozilla-central@d6d6fd889f7b [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersglandium
bugs1401725
milestone57.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1401725 - Use sparse checkouts for toolchain tasks; r=glandium Strictly speaking we don't need all files in the directories listed in the profile. But the checkout is still small enough and it is far less effort than cherry-picking every file needed by every toolchain task. This brings the checkout down to ~3700 files, which only takes 1-2s. MozReview-Commit-ID: 2BpKdZ2Pvx9
build/sparse-profiles/toolchain-build
taskcluster/taskgraph/transforms/job/toolchain.py
new file mode 100644
--- /dev/null
+++ b/build/sparse-profiles/toolchain-build
@@ -0,0 +1,9 @@
+# Profile needed to build toolchain tasks.
+
+# This is probably a little wider than we need it to be. But it is
+# still relatively small and it keeps this profile simple.
+%include build/sparse-profiles/taskgraph
+
+[include]
+# Needed by build-clang.py.
+path:tools/rewriting/
\ No newline at end of file
--- a/taskcluster/taskgraph/transforms/job/toolchain.py
+++ b/taskcluster/taskgraph/transforms/job/toolchain.py
@@ -101,33 +101,34 @@ def docker_worker_toolchain(config, job,
 
     worker = taskdesc['worker']
     worker['artifacts'] = []
     worker['chain-of-trust'] = True
 
     docker_worker_add_public_artifacts(config, job, taskdesc)
     docker_worker_add_tc_vcs_cache(config, job, taskdesc)
     docker_worker_add_gecko_vcs_env_vars(config, job, taskdesc)
-    support_vcs_checkout(config, job, taskdesc)
+    support_vcs_checkout(config, job, taskdesc, sparse=True)
 
     env = worker['env']
     env.update({
         'MOZ_BUILD_DATE': config.params['moz_build_date'],
         'MOZ_SCM_LEVEL': config.params['level'],
         'TOOLS_DISABLE': 'true',
         'MOZ_AUTOMATION': '1',
     })
 
     if run['tooltool-downloads']:
         internal = run['tooltool-downloads'] == 'internal'
         docker_worker_add_tooltool(config, job, taskdesc, internal=internal)
 
     worker['command'] = [
         '/builds/worker/bin/run-task',
         '--vcs-checkout=/builds/worker/workspace/build/src',
+        '--sparse-profile', 'build/sparse-profiles/toolchain-build',
         '--',
         'bash',
         '-c',
         'cd /builds/worker && '
         './workspace/build/src/taskcluster/scripts/misc/{}'.format(
             run['script'])
     ]