author Martin Thomson <>
Mon, 03 Sep 2018 16:31:31 +1000
changeset 14493 e56241dc5b61300dc426884b7c3cf50f28e9586b
parent 12998 1c386a0c7bcac4af17ddda80c2acacbb4fe8f40e
child 15156 0c9d61532c3c528b325d15e8044ccebffe7b62ed
permissions -rw-r--r--
Bug 1488148 - Rework CI images, r=jcj This does some fairly major restructuring of the docker images we use for CI. The genesis of the change is that we were pulling a version of clang that didn't work for fuzzing tests. It turns out that is a use case that is not well-supported anyway, and we have open bugs on it, but this installs workarounds for all the problems I found. Firstly, our images were bloated. This slims down most of the images. The biggest gains are in the clang-format image (down to around a fifth of its previous size). The main linux image we use for building and running tests is also less than half its original size. To achieve that, I had to make two new images. One for all the esoteric builds we run (we compile with multiple gcc and clang versions, and I've added some more to that list). That's a fairly sizeable image. The other is for the interop and bogo suites, where we rely on having Rust and go available. go adds a tidy 250Mb to an image, and Rust adds 750Mb. Using an image with both of those for regular builds can't be good for performance. I didn't expect to see real performance gains here, but the Linux build (32-bit, default config) went from 4:18 down to 2:42 (roughly). The bulk of that time is accounted for by downloading the docker image, so it's clear that an optimization worth spending the time on. Secondly, we had a lot of custom configuration stuff in the builds. This removes most of that in favour of using stock Ubuntu packages from 18.04. The one exception here is - I hope - temporary. As noted in the bug comments, the current release of LLVM 6 has a bug where you can't run address sanitizer on a 32-bit machine if it has glibc 2.27 (which Ubuntu 18.04 does). That's fairly crippling because we need a newer version of LLVM than runs by default on Ubuntu 16.04, so we're stuck with installing a non-stock version for 32-bit runs. I've opted to (temporarily) run 16.04 with an LLVM from the LLVM project. The final change, which is minor, but a little odd and worth noting: the images now rely on "localhost.localdomain" being aliased to the local machine. This is something :wcosta has done for us (thanks!). Thus, we no longer have to run as root so that we can tweak /etc/hosts before we run. There is a little cleanup related to this, but nothing significant. (The scripts still include the `su worker` tweak for aarch64, but I've added a guard and we can remove that with bug 1488325.) There is still more work to be done for the HACL* and SAW builds, which use some very strange configurations. Also, all of the aarch64 images aren't built automatically, so we use images from Franziskus' dockerhub account. This is not good. After digging around a little, there's probably something to be done with QEMU, but I decided that was a project for another time.

# Dockerfile for running fuzzing tests.
# Note that when running this, you need to add `--cap-add SYS_PTRACE` to the
# docker invocation or ASAN won't work.
# On taskcluster use `features: ["allowPtrace"]`.
# See
FROM ubuntu:18.04
LABEL maintainer="Martin Thomson <>"

RUN dpkg --add-architecture i386
RUN apt-get update \
 && apt-get install -y --no-install-recommends \
    build-essential \
    ca-certificates \
    clang \
    clang-tools \
    curl \
    g++-multilib \
    git \
    gyp \
    libssl-dev \
    libssl-dev:i386 \
    libxml2-utils \
    lib32z1-dev \
    linux-libc-dev:i386 \
    llvm-dev \
    locales \
    mercurial \
    ninja-build \
    pkg-config \
    valgrind \
    zlib1g-dev \
 && rm -rf /var/lib/apt/lists/* \
 && apt-get autoremove -y && apt-get clean -y

ENV SHELL /bin/bash
ENV USER worker
ENV HOST localhost
ENV DOMSUF localdomain

RUN locale-gen $LANG \
 && DEBIAN_FRONTEND=noninteractive dpkg-reconfigure locales

RUN useradd -d $HOME -s $SHELL -m $USER

# Add build and test scripts.
ADD bin $HOME/bin
RUN chmod +x $HOME/bin/*

# Change user.

# Set a default command for debugging.
CMD ["/bin/bash", "--login"]