Bug 1372892: make tasks with optimized dependencies depend on decision task; r=aki
authorDustin J. Mitchell <dustin@mozilla.com>
Sat, 01 Jul 2017 17:01:42 -0400
changeset 420777 bf3ae82e04e69085be7f226b88917f3f36e0677b
parent 420776 b790837d862c627c3ca4af120653bea197513fcd
child 420778 e0029dcf7b3eb7c1ab8db350c3d2b9fb2feadc3a
push id7566
push usermtabara@mozilla.com
push dateWed, 02 Aug 2017 08:25:16 +0000
treeherdermozilla-beta@86913f512c3c [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersaki
bugs1372892, 1372817
milestone56.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1372892: make tasks with optimized dependencies depend on decision task; r=aki This addresses the issue where tasks with a dependency which is already complete started immediately, without waiting for the decision task (bug 1372817). It does not address the issue where a re-run of a failed decision task will allow tasks created by the first run to run, in addition to creating an entirely new set of tasks. MozReview-Commit-ID: EdNZSrNw3F6
taskcluster/taskgraph/create.py
--- a/taskcluster/taskgraph/create.py
+++ b/taskcluster/taskgraph/create.py
@@ -53,21 +53,25 @@ def create_tasks(taskgraph, label_to_tas
         # Using visit_postorder() here isn't the most efficient: we'll
         # block waiting for dependencies of task N to submit even though
         # dependencies for task N+1 may be finished. If we need to optimize
         # this further, we can build a graph of task dependencies and walk
         # that.
         for task_id in taskgraph.graph.visit_postorder():
             task_def = taskgraph.tasks[task_id].task
             attributes = taskgraph.tasks[task_id].attributes
-            # if this task has no dependencies, make it depend on this decision
-            # task so that it does not start immediately; and so that if this loop
-            # fails halfway through, none of the already-created tasks run.
-            if decision_task_id and not task_def.get('dependencies'):
-                task_def['dependencies'] = [decision_task_id]
+
+            # if this task has no dependencies *within* this taskgraph, make it
+            # depend on this decision task. If it has another dependency within
+            # the taskgraph, then it already implicitly depends on the decision
+            # task.  The result is that tasks do not start immediately. if this
+            # loop fails halfway through, none of the already-created tasks run.
+            if decision_task_id:
+                if not any(t in taskgraph.tasks for t in task_def.get('dependencies', [])):
+                    task_def.setdefault('dependencies', []).append(decision_task_id)
 
             task_def['taskGroupId'] = task_group_id
             task_def['schedulerId'] = scheduler_id
 
             # Wait for dependencies before submitting this.
             deps_fs = [fs[dep] for dep in task_def.get('dependencies', [])
                        if dep in fs]
             for f in futures.as_completed(deps_fs):