2070734d3a09fe05a77ef3c258f67018de597db8: Bug 1427336 - Fix recursive adding of directories through %include in Dockerfiles. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Fri, 29 Dec 2017 14:42:14 +0900 - rev 714733
Push 94017 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 05:49:20 +0000
Bug 1427336 - Fix recursive adding of directories through %include in Dockerfiles. r?dustin Giving a directory to %include would copy all leaf files under one single directory in the context image. The only image affected is valgrind-build, which ended up having a dot-config/pip.conf file instead of dot-config/pip/pip.conf, meaning valgrind jobs weren't using the pip config.
391c87d9911cb12c498a43a1e38a8b02ce901b31: Bug 1419638 - Allow to share docker image definitions. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Sun, 24 Dec 2017 07:58:08 +0900 - rev 714732
Push 94016 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 05:22:30 +0000
Bug 1419638 - Allow to share docker image definitions. r?dustin Instead of duplicating Dockerfiles between taskcluster/docker/* directories, which can be error prone for very close images, it can be desirable to use the same file. This change allows to set the `definition` keyword on a docker image definition in kind.yml that will make the task use the files from taskcluster/docker/<definition> instead of taskcluster/docker/<image_name>.
37b623a958aea9fa83c1497163855d7e327de71b: Bug 1427316 - Check differences between builds with and without the changes. draft
Mike Hommey <mh+mozilla@glandium.org> - Fri, 29 Dec 2017 11:13:46 +0900 - rev 714731
Push 94015 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 04:46:41 +0000
Bug 1427316 - Check differences between builds with and without the changes. This is not meant to land. This is only for documentation.
45fdc26a4fa34b812654962745f4d311e0571aca: Bug 1427316 - Use the binutils we just built to build GCC. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Fri, 29 Dec 2017 09:33:14 +0900 - rev 714730
Push 94015 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 04:46:41 +0000
Bug 1427316 - Use the binutils we just built to build GCC. r?build We're currently building GCC with the system binutils, which, at the moment, is whatever version is available on the CentOS 6 build environments. With the imminent switch to Debian 7, that will be a different version. It turns out the GCC configure script does enable some features depending on the binutils it's built with. For the most notable differences it makes when going from Centos 6 to Debian, it enables .init_array/.fini_array depending on the binutils version, and enables the use of CFI advances depending on gas and objdump respectively supporting and displaying DW_CFA_advance_loc. But we're already building a fixed version of binutils (which happens to be more recent than the one in both CentOS 6 and Debian 7), and we're using that version when using GCC to build, so we can just as much use the version we built to build GCC. In order to avoid any changes to the resulting builds, we explicitly turn off .init_array/.fini_array (which currently happens implicitly when building on CentOS 6). This will ensure that there is not other change to the builds due to this binutils version bump (.init_array/.fini_array being enabled shifts everything in the binaries, so it makes the whole diff full of noise)
07dd89d8cc4d91c8d5470fd2f1fb0bb8fc19304e: Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 12:14:34 +0900 - rev 714729
Push 94015 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 04:46:41 +0000
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin There are e.g. some build infrastructure changes that we want to have a controlled impact on the Firefox builds we produce. We have, in multiple occasions, gone through manual work to compare Firefox builds, most of the time using the diffoscope tool (https://diffoscope.org/). This change introduces a new task kind that takes two Firefox builds as input, either by name (reference to a build from the current task graph) or by index (reference to a build from a previous push), and compares them. In order to get a Firefox build by index, we rely on dummy tasks with an optimization we expect to always hit, so we add the necessary bits to ensure those dummy tasks can go through up to the optimization phase and be optimized out there.
1b6b6203dbb869f83f87fd478c43f7a116675b33: Bug 1427266 - Statically link libdmg-hfsplus against OpenSSL. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 18:19:31 +0900 - rev 714728
Push 94015 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 04:46:41 +0000
Bug 1427266 - Statically link libdmg-hfsplus against OpenSSL. r?build libcrypto, part of OpenSSL, and that dmg links against, has a varying ABI, and something built against libcrypto on Centos won't run on Debian and vice versa. It might not even work between versions of the same OS (e.g. Debian 7 vs. Debian 9). Because of that, it is desirable to statically link it. This incorporates https://github.com/mozilla/libdmg-hfsplus/pull/1 and sets OPENSSL_USE_STATIC_LIBS when building libdmg-hfsplus.
a9a0b36df50247a0242bac57b701ec67b2adcf16: Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 12:14:34 +0900 - rev 714727
Push 94014 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:44:17 +0000
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin There are e.g. some build infrastructure changes that we want to have a controlled impact on the Firefox builds we produce. We have, in multiple occasions, gone through manual work to compare Firefox builds, most of the time using the diffoscope tool (https://diffoscope.org/). This change introduces a new task kind that takes two Firefox builds as input, either by name (reference to a build from the current task graph) or by index (reference to a build from a previous push), and compares them. In order to get a Firefox build by index, we rely on dummy tasks with an optimization we expect to always hit, so we add the necessary bits to ensure those dummy tasks can go through up to the optimization phase and be optimized out there.
92cb45537af0bc6e6b345a9c50eb34db7aefe415: Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 12:14:34 +0900 - rev 714726
Push 94013 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:39:29 +0000
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin There are e.g. some build infrastructure changes that we want to have a controlled impact on the Firefox builds we produce. We have, in multiple occasions, gone through manual work to compare Firefox builds, most of the time using the diffoscope tool (https://diffoscope.org/). This change introduces a new task kind that takes two Firefox builds as input, either by name (reference to a build from the current task graph) or by index (reference to a build from a previous push), and compares them. In order to get a Firefox build by index, we rely on dummy tasks with an optimization we expect to always hit, so we add the necessary bits to ensure those dummy tasks can go through up to the optimization phase and be optimized out there.
cab12eef160096e2a02264b936c26dd278f3468d: Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 12:14:34 +0900 - rev 714725
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427312 - Add mechanism to create tasks to compare Firefox builds. r?dustin There are e.g. some build infrastructure changes that we want to have a controlled impact on the Firefox builds we produce. We have, in multiple occasions, gone through manual work to compare Firefox builds, most of the time using the diffoscope tool (https://diffoscope.org/). This change introduces a new task kind that takes two Firefox builds as input, either by name (reference to a build from the current task graph) or by index (reference to a build from a previous push), and compares them. In order to get a Firefox build by index, we rely on dummy tasks with an optimization we expect to always hit, so we add the necessary bits to ensure those dummy tasks can go through up to the optimization phase and be optimized out there.
6bb4453e3e015e79ac2c8267d7488c83fc20a534: Statically link libdmg-hfsplus against openssl. draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 18:19:31 +0900 - rev 714724
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Statically link libdmg-hfsplus against openssl. There is a OPENSSL_USE_STATIC_LIBS define, but dmg's CMakeLists.txt doesn't respect it, so patch it so that it does.
82083aaa3c15920072b9af41055bd3c5b027889e: Bug 1427232 - Install expat-devel before building gtk3. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 11:23:01 +0900 - rev 714723
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427232 - Install expat-devel before building gtk3. r?build fontconfig uses expat by default to read its xml configuration, but when expat is not there at build time, it falls back to libxml2. We ended up in this situation, while on Debian, fontconfig is built against expat. This makes no practical difference, since we're not actually using fontconfig, but for some reason, the difference in dependencies has an impact on how ld chooses to arrange the .plt and .got.plt sections, meaning that even though the code and data is originally identical, in the .o files, the resulting linked machine code is largely different because of all the applied relocations changing the offsets of e.g. call instructions for function calls through the .plt. This results in large differences in .plt, .init, .text, .data.rel.ro, as well as symbols list when building on Debian. This thus is meant to help make the differences more tractable.
6813481f2f7d3a2041763e3514768d777be020fc: Bug 1427232 - Remove .la files when building gtk3. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 11:15:00 +0900 - rev 714722
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427232 - Remove .la files when building gtk3. r?build In bug 1426785, Gtk+3 build was moved to docker image creation time, and at the same time, the removal of .la files was stripped, because it seemed too broad to do that in /usr/local/lib*, and because I didn't really remember why it was there in the first place. I now do remember, and that's because libtool likes to add useless dependencies on libraries just because direct dependencies transitively depend on them. For example, this adds a dependency on libffi to libpangocairo, which doesn't use it directly. I also happens that /usr/local/lib* is empty at the moment we build gtk3, so we can just do the cleanup there. On its own, this change is useless, but: - it restores the libraries in their state pre-bug 1426785, - it helps reduce some differences when building on Debian (for bug 1399679), easing the comparison of those builds.
f1494990a4ad94bce28b0bda6bd2dcf9b2ac71a7: Bug 1419638 - Allow to share docker image definitions. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Sun, 24 Dec 2017 07:58:08 +0900 - rev 714721
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1419638 - Allow to share docker image definitions. r?dustin Instead of duplicating Dockerfiles between taskcluster/docker/* directories, which can be error prone for very close images, it can be desirable to use the same file. This change allows to set the `definition` keyword on a docker image definition in kind.yml that will make the task use the files from taskcluster/docker/<definition> instead of taskcluster/docker/<image_name>.
ff880f4cf5c01e4bde6a0c653da89a4c89eb8d3c: Bug 1419638 - Allow to pass arguments to docker when building docker-images. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Sun, 24 Dec 2017 07:51:29 +0900 - rev 714720
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1419638 - Allow to pass arguments to docker when building docker-images. r?dustin Ideally, we'd simply use the --build-arg docker argument along with ARG in the Dockerfile, but that's only supported from Docker API 1.21, and we're stuck on 1.18 for the moment. So we add another hack to how we handle the Dockerfile, by adding a commented syntax that allows to declare arguments to the Dockerfile. The arguments can be defined in the docker images kind.yml file through the `args` keyword. Under the hood, they are passed down to the docker image task through the environment. The mach taskcluster-build-image command then uses the corresponding values from the environment to generate a "preprocessed" Dockerfile for its context.
ef843af69aefd6238aca2446ef3fa4ed2f69d9c6: Bug 1419638 - Add schema validation to docker image transform. r?dustin draft
Mike Hommey <mh+mozilla@glandium.org> - Thu, 28 Dec 2017 15:46:14 +0900 - rev 714719
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1419638 - Add schema validation to docker image transform. r?dustin
393455f8b44f90abc7af334c11226bd60b88c20a: Bug 1427155 - Don't set STRIP_FLAGS from in-tree mozconfigs. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Wed, 27 Dec 2017 15:44:12 +0900 - rev 714718
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427155 - Don't set STRIP_FLAGS from in-tree mozconfigs. r?build The reason it was set from mozconfigs is that profiling require it. But since it was added, bug 751355 made it implied by --enable-profiling, and bug 1144842 further made sure that profiling and STRIP_FLAGS were tied together.
d8da4631c5e61d12366a1c1e2c832e4d2046ba3f: Bug 1427150 - Don't get some default sink info based on the pa_sink_info build-time size. r?padenot draft
Mike Hommey <mh+mozilla@glandium.org> - Wed, 27 Dec 2017 17:31:57 +0900 - rev 714717
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427150 - Don't get some default sink info based on the pa_sink_info build-time size. r?padenot The build-time pa_sink_info might not have the same size as the at run time. This especially becomes a problem when the struct is larger at build-time, since we copy its content for the default sink info. However, we don't need most of the data in there, so create a new struct holding what we do need, and copy that. This new struct has a constant size across versions of pulseaudio.
22b2391379f6ba555e31769949be0fa39f76657b: Bug 1427150 - Don't read pa_sink_port_info.available when runtime pulseaudio is < 2.0. r?padenot draft
Mike Hommey <mh+mozilla@glandium.org> - Wed, 27 Dec 2017 14:45:53 +0900 - rev 714716
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427150 - Don't read pa_sink_port_info.available when runtime pulseaudio is < 2.0. r?padenot The code reading that field is build-time protected, so as long as the runtime pulseaudio version is greater than the one used at build-time, we're good. And as of now, the build-time version is 0.9.something, from our Centos 6 build environment. In bug 1399679, we're going to switch the build environment to Debian Wheezy, which comes with pulseaudio 2.0 headers, enabling that code. This means running the official Firefox on e.g. Ubuntu 12.04, which has 1.1, would read past the end of the pa_sink_port_info, possibly leading to random behavior with port availability.
b6a9c857584309a3a700e7c52614b85c7cc97847: Bug 1427145 - Use toolchain artifacts instead of tooltool packages for osx (cross) repackages. r?build draft
Mike Hommey <mh+mozilla@glandium.org> - Wed, 27 Dec 2017 07:02:21 +0900 - rev 714715
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427145 - Use toolchain artifacts instead of tooltool packages for osx (cross) repackages. r?build OSX (cross) repackages are currently using a tooltool manifest to get libdmg and hfsplus. Change those jobs to use the toolchain artifacts instead. At the same time, modify the repackage mozharness script's _run_tooltool so that it doesn't fail with MOZ_TOOLCHAINS being set but without a tooltool_manifest_src, matching the similar function in buildbase.py.
f8ea2f512979f6877dbaa927cc14bf4bb271e3dc: Bug 1427068 - Enable the mercurial share extension at the system level. r?gps draft
Mike Hommey <mh+mozilla@glandium.org> - Tue, 26 Dec 2017 17:01:55 +0900 - rev 714714
Push 94012 by bmo:mh+mozilla@glandium.org at Fri, 29 Dec 2017 01:24:43 +0000
Bug 1427068 - Enable the mercurial share extension at the system level. r?gps I suppose it was setup through ~worker/.hgrc before we started installing a /etc/mercurial/hgrc that enables a few other extensions and sets some preferences. There is no reason to now have two places where mercurial is being set up, and it feels natural that we set it up at the system level. Ideally, we'd also clean up the centos6-based images, but they require an update of the centos6-build and centos6-build-upd images on the docker hub, which is not really convenient, and those images are going to be obsoleted soon anyways (bug 1399679).
(0) -300000 -100000 -30000 -10000 -3000 -1000 -300 -100 -50 -20 +20 +50 +100 +300 +1000 +3000 +10000 +30000 +100000 tip