Bug 1555032 - [taskgraph] Add ability to pass external optimize strategies via env, r=tomprince
authorAndrew Halberstadt <ahalberstadt@mozilla.com>
Fri, 31 May 2019 12:56:47 +0000
changeset 476362 3a5ec7efe864b5cd3e351716896e6da017830b76
parent 476361 4eaea9067bae125fe98db1a76527256636da410e
child 476363 70d1aed471491825c66bb5da924dd88006445ab3
push id36092
push userarchaeopteryx@coole-files.de
push dateFri, 31 May 2019 17:03:46 +0000
treeherdermozilla-central@8384972e1f6a [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewerstomprince
bugs1555032
milestone69.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1555032 - [taskgraph] Add ability to pass external optimize strategies via env, r=tomprince This allows users to set TASKGRAPH_OPTIMIZE_STRATEGIES to a python_path.find_object string. E.g: TASKGRAPH_OPTIMIZE_STRATEGIES="module:strategies" ./mach taskgraph optimized This opens the door to swap in external strategies at runtime and will be used for back testing experimental strategies. Differential Revision: https://phabricator.services.mozilla.com/D33203
taskcluster/taskgraph/generator.py
--- a/taskcluster/taskgraph/generator.py
+++ b/taskcluster/taskgraph/generator.py
@@ -318,20 +318,29 @@ class TaskGraphGenerator(object):
             target_graph)
         yield verifications('target_task_graph', target_task_graph, graph_config)
 
         logger.info("Generating optimized task graph")
         existing_tasks = parameters.get('existing_tasks')
         do_not_optimize = set(parameters.get('do_not_optimize', []))
         if not parameters.get('optimize_target_tasks', True):
             do_not_optimize = set(target_task_set.graph.nodes).union(do_not_optimize)
-        optimized_task_graph, label_to_taskid = optimize_task_graph(target_task_graph,
-                                                                    parameters,
-                                                                    do_not_optimize,
-                                                                    existing_tasks=existing_tasks)
+
+        # this is used for testing experimental optimization strategies
+        strategies = os.environ.get('TASKGRAPH_OPTIMIZE_STRATEGIES')
+        if strategies:
+            strategies = find_object(strategies)
+
+        optimized_task_graph, label_to_taskid = optimize_task_graph(
+            target_task_graph,
+            parameters,
+            do_not_optimize,
+            existing_tasks=existing_tasks,
+            strategies=strategies,
+        )
 
         yield verifications('optimized_task_graph', optimized_task_graph, graph_config)
 
         morphed_task_graph, label_to_taskid = morph(
             optimized_task_graph, label_to_taskid, parameters)
 
         yield 'label_to_taskid', label_to_taskid
         yield verifications('morphed_task_graph', morphed_task_graph, graph_config)