Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; r=dustin
authorGregory Szorc <gps@mozilla.com>
Fri, 29 Jul 2016 10:09:06 -0700
changeset 307252 f351e9f3e564f2f3f025664e1f701a151c61e1bb
parent 307251 f33cb032cc9f1f01b22f6663c1553a6e481e7f59
child 307253 76de1a35ddb7c4ec9ec33adc97d3abb87ddb5c78
push id30927
push userdmitchell@mozilla.com
push dateFri, 29 Jul 2016 17:32:52 +0000
treeherderautoland@f351e9f3e564 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersdustin
bugs1290516
milestone50.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1290516 - Remove {{project}} from cache path for decision and action tasks; r=dustin Currently, the gecko-decision and action tasks maintain a separate cache for each "project" (Firefox repo) source checkout. In my opinion, this is not necessary. Separate caches/source checkouts mean a lower chance for a cache hit: a lower chance that any given worker will have a source checkout readily available for the "project" being built. They also mean workers store more data and files in caches. By using a shared cache for source checkouts, workers will ideally only have a single source checkout cache and will be pretty much guaranteed to have a checkout readily available (the main exception being the first task on a freshly-provisioned worker). A downside of a shared cache is that workers may spend more time doing `hg update` operations. e.g. if the last task was from mozilla-release and the current task is from mozilla-central, thousands of files will be updated. However, I believe we will still be better off because separate caches *increase* the probability for a full checkout, which involves writing ~150,000 files! So I believe this change will lower the overall amount of VCS work done by workers while possibly sacrificing the lowest possible execution time (which is achieved by minimizing the `hg update` work). It's worth noting that `hg robustcheckout` (which is used by these tasks) does the equivalent of an `hg purge --all`. This ensures that all untracked and ignored files from previous tasks are deleted. So there should be no contamination from previous tasks. MozReview-Commit-ID: I5Ng3xFEfou
.taskcluster.yml
taskcluster/taskgraph/action.yml
--- a/.taskcluster.yml
+++ b/.taskcluster.yml
@@ -82,17 +82,17 @@ tasks:
             --base-repository='https://hg.mozilla.org/mozilla-central'
             --head-repository='{{{url}}}'
             --head-ref='{{revision}}'
             --head-rev='{{revision}}'
             --revision-hash='{{revision_hash}}'
 
         cache:
           level-{{level}}-hg-shared: /home/worker/hg-shared
-          level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+          level-{{level}}-gecko-decision: /home/worker/workspace
 
         features:
           taskclusterProxy: true
 
         # Note: This task is built server side without the context or tooling that
         # exist in tree so we must hard code the version
         image: 'taskcluster/decision:0.1.2'
 
--- a/taskcluster/taskgraph/action.yml
+++ b/taskcluster/taskgraph/action.yml
@@ -32,17 +32,17 @@ payload:
     GECKO_HEAD_REF: '{{head_ref}}'
     GECKO_HEAD_REV: '{{head_rev}}'
     ACTION_ARGS: >
       --decision-id='{{decision_task_id}}'
       --task-labels='{{task_labels}}'
 
   cache:
     level-{{level}}-hg-shared: /home/worker/hg-shared
-    level-{{level}}-{{project}}-gecko-decision: /home/worker/workspace
+    level-{{level}}-gecko-decision: /home/worker/workspace
 
   features:
     taskclusterProxy: true
 
   # Note: This task is built server side without the context or tooling that
   # exist in tree so we must hard code the version
   image: 'taskcluster/decision:0.1.2'