Bug 1390700 - Use sparse checkouts for decision task; r=dustin
☠☠ backed out by 874e11b5b24e ☠ ☠
authorGregory Szorc <gps@mozilla.com>
Wed, 23 Aug 2017 12:09:00 -0700
changeset 426229 9923fffd4f64a1aa9d762e6027e0e2424a19c49c
parent 426228 d243323c86865bf7d6cef0650db71ffce3ee941c
child 426230 0516a2ab29450623e90897b732cf3ab4884a67a6
push id7761
push userjlund@mozilla.com
push dateFri, 15 Sep 2017 00:19:52 +0000
treeherdermozilla-beta@c38455951db4 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersdustin
bugs1390700
milestone57.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1390700 - Use sparse checkouts for decision task; r=dustin The updated Docker image contains robustcheckout and run-task support for sparse checkouts, which are obvious prerequisites. We change the cache name so sparse and non-sparse checkouts don't use the same working directory. If we didn't do this, tasks running from images with old Mercurial clients or without a sparse aware robustcheckout would fail. The effect of using a sparse checkout is that we reduce the number of files in the checkout from ~234,000 to ~3,600. This reduces time for a fresh checkout from several dozen seconds to under 2s. MozReview-Commit-ID: IJz794g8ZKH
.taskcluster.yml
taskcluster/docker/decision/HASH
taskcluster/docker/decision/VERSION
--- a/.taskcluster.yml
+++ b/.taskcluster.yml
@@ -68,37 +68,39 @@ tasks:
         # checkout-gecko uses these to check out the source; the inputs
         # to `mach taskgraph decision` are all on the command line.
         GECKO_BASE_REPOSITORY: 'https://hg.mozilla.org/mozilla-unified'
         GECKO_HEAD_REPOSITORY: '${repoUrl}'
         GECKO_HEAD_REF: '${push.revision}'
         GECKO_HEAD_REV: '${push.revision}'
         GECKO_COMMIT_MSG: '${push.comment}'
         HG_STORE_PATH: /home/worker/checkouts/hg-store
+        TASKCLUSTER_CACHES: /home/worker/checkouts
 
       cache:
-        level-${repository.level}-checkouts: /home/worker/checkouts
+        level-${repository.level}-checkouts-sparse-v1: /home/worker/checkouts
 
       features:
         taskclusterProxy: true
         chainOfTrust: true
 
       # Note: This task is built server side without the context or tooling that
       # exist in tree so we must hard code the hash
       # XXX Changing this will break Chain of Trust without an associated puppet and
       # scriptworker patch!
-      image: 'taskcluster/decision:0.1.9@sha256:9d6a38ff675ba2f2f132bc1461e3a48b0ce56c90617a8a5c5f8c48c6eb8d06e0'
+      image: 'taskcluster/decision:0.1.10@sha256:c5451ee6c655b3d97d4baa3b0e29a5115f23e0991d4f7f36d2a8f793076d6854'
 
       maxRunTime: 1800
 
       # TODO use mozilla-unified for the base repository once the tc-vcs
       # tar.gz archives are created or tc-vcs isn't being used.
       command:
         - /home/worker/bin/run-task
         - '--vcs-checkout=/home/worker/checkouts/gecko'
+        - '--sparse-profile=build/sparse-profiles/taskgraph'
         - '--'
         - bash
         - -cx
         - $let:
             extraArgs: {$if: 'tasks_for == "hg-push"', then: '', else: '${cron.quoted_args}'}
           # NOTE: the explicit reference to mozilla-central below is required because android-stuff
           # still uses tc-vcs, which does not support mozilla-unified
           # https://bugzilla.mozilla.org/show_bug.cgi?id=1383973
--- a/taskcluster/docker/decision/HASH
+++ b/taskcluster/docker/decision/HASH
@@ -1,1 +1,1 @@
-sha256:9d6a38ff675ba2f2f132bc1461e3a48b0ce56c90617a8a5c5f8c48c6eb8d06e0
+sha256:c5451ee6c655b3d97d4baa3b0e29a5115f23e0991d4f7f36d2a8f793076d6854
--- a/taskcluster/docker/decision/VERSION
+++ b/taskcluster/docker/decision/VERSION
@@ -1,1 +1,1 @@
-0.1.9
+0.1.10