Bug 1428907 - Only use packages task digests to compute docker image digests. r=dustin
authorMike Hommey <mh+mozilla@glandium.org>
Tue, 09 Jan 2018 07:31:56 +0900
changeset 450232 52f9649d3d8f32d71805c1ff9243d854a947c9ee
parent 450231 c535f544f4bc541242071ee11c268ebc74dc4113
child 450233 5786d2b246ee6cb20309b41cc1fba25c33953bcf
push id8527
push userCallek@gmail.com
push dateThu, 11 Jan 2018 21:05:50 +0000
treeherdermozilla-beta@95342d212a7a [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersdustin
bugs1428907, 1427326
milestone59.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1428907 - Only use packages task digests to compute docker image digests. r=dustin In bug 1427326, we added package tasks that can be depended upon by docker image tasks. In that case, we add the routes containing a digest for those package tasks when computing the docker image digests. The problem is that those routes start with 'index.gecko.cache.level-n' where n varies between try and e.g. mozilla-central. That means the digest for those docker images varies between try and e.g. mozilla-central, which then prevents try from using the cached versions for mozilla-central when there is one, like for other docker images without package dependencies. What we really need from those routes is the digest part, which is independent of the level, and we don't actually care about anything else in the route string, so just use the digest.
taskcluster/taskgraph/transforms/docker_image.py
--- a/taskcluster/taskgraph/transforms/docker_image.py
+++ b/taskcluster/taskgraph/transforms/docker_image.py
@@ -1,15 +1,16 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
 from __future__ import absolute_import, print_function, unicode_literals
 
 import os
+import re
 
 from taskgraph.transforms.base import TransformSequence
 from .. import GECKO
 from taskgraph.util.docker import (
     docker_image,
     generate_context_hash,
 )
 from taskgraph.util.cached_tasks import add_optimization
@@ -17,16 +18,18 @@ from taskgraph.util.schema import (
     Schema,
     validate_schema,
 )
 from voluptuous import (
     Optional,
     Required,
 )
 
+DIGEST_RE = re.compile('^[0-9a-f]{64}$')
+
 transforms = TransformSequence()
 
 docker_image_schema = Schema({
     # Name of the docker image.
     Required('name'): basestring,
 
     # Treeherder symbol.
     Required('symbol'): basestring,
@@ -60,17 +63,20 @@ def validate(config, tasks):
 def fill_template(config, tasks):
     available_packages = {}
     for task in config.kind_dependencies_tasks:
         if task.kind != 'packages':
             continue
         name = task.label.replace('packages-', '')
         for route in task.task.get('routes', []):
             if route.startswith('index.') and '.hash.' in route:
-                available_packages[name] = route
+                # Only keep the hash part of the route.
+                h = route.rsplit('.', 1)[1]
+                assert DIGEST_RE.match(h)
+                available_packages[name] = h
                 break
     for task in tasks:
         image_name = task.pop('name')
         job_symbol = task.pop('symbol')
         args = task.pop('args', {})
         definition = task.pop('definition', image_name)
         packages = task.pop('packages', [])