ed95bb78b383b92d422d83ff0b26a2aaf087c5d8: Bug 1290531 - Move tooltool.py into shared directory; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:50:24 -0700 - rev 307537
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Move tooltool.py into shared directory; r=dustin The reason tooltool.py was vendored in testing/docker/decision was because locally built Docker images were using vanilla `docker build` and didn't know about our special Dockerfile syntax to allow the inclusion of images from outside the directory where the Dockerfile was located. Now that locally-built Docker images know of our special Dockerfile syntax, we can include files from anywhere. So, move tooltool.py to a shared directory, away from the decision image. I didn't bump the version of the decision image because there are a few more things I want to do to this image, such as have it use the `checkout-gecko-and-run` script instead of its own script. I think I'll do that in a separate bug, however. MozReview-Commit-ID: 12ze9GJFMJq
d61de5431643721093c23fc70f3a52c62fd3f626: Bug 1290531 - Build Docker images from custom tar contexts; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:41:59 -0700 - rev 307536
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Build Docker images from custom tar contexts; r=dustin Now that Docker image building is called from Python, we can start to do advanced stuff with it. With this commit, we switch from building Docker images directly from the source directory ("the Docker way") to using our custom Docker image build contexts. The main advantage of this is that locally-built Docker images can now use our custom Dockerfile syntax to include extra files in the build context! The code for building a Docker image from a context has been extracted to its own standalone function. I have nefarious plans for this in the future, such as the ability to override the FROM syntax to specify URLs of images. This would allow us to host base images on our own server, which removes a dependency on Docker Hub and improves determinism, since images on Docker Hub change all the time. MozReview-Commit-ID: 5lTdV8yEHkc
4ec50d432877c8dc9cfdd9b8192a125533ae7e64: Bug 1290531 - Invoke docker from Python, remove build.sh; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:22:06 -0700 - rev 307535
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Invoke docker from Python, remove build.sh; r=dustin build.sh had been reduced to invoking `docker`. We move that invocation to Python and remove build.sh. Long live build.sh! MozReview-Commit-ID: FQBDJv4HSaU
9b290fa0e180d016c95441b880c64e07c424e7ab: Bug 1290531 - Remove support for building with custom build.sh; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:18:27 -0700 - rev 307534
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Remove support for building with custom build.sh; r=dustin Now that tester-device is gone, there are no more images using custom build.sh scripts and that feature can be deleted. Yay simplicity. MozReview-Commit-ID: 5JqFZ2HOsWs
1532b022e19cf3f6b483649fb6dccbbae7a5226f: Bug 1290531 - Remove tester-device Docker image; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:17:16 -0700 - rev 307533
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Remove tester-device Docker image; r=dustin There are no references to tester-device in tree or in the gaia repo. Since it appears to be unused, remove it. According to garndt, this was used for testing in the remote device lab, which he things is no longer happening. The reason I found this is because it is the only thing using a custom "build.sh" to create Docker images. I'm rewriting the Docker image building functionality and tester-device is a one-off interfering with that work. Making it go away is the easiest way to unblock me. MozReview-Commit-ID: 4veem4uIXXi
e20bc9e7b3887a48d32e7fb242e94986555c31f6: Bug 1290531 - Move image tag resolution to Python; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 13:06:10 -0700 - rev 307532
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Move image tag resolution to Python; r=dustin We already had code for resolving the image registry and tag. We refactored it slightly to be more useful then changed build.sh to accept the tag as an argument. At this point, build.sh is basically a wrapper around `docker`. But there's a special case for executing custom "build.sh" files we need to eliminate first... MozReview-Commit-ID: A9HVvxgCdG2
a1f4555494b7aa5ebc6dcdae45315e1ac2ab0bb4: Bug 1290531 - Move image name verification to Python; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 12:59:46 -0700 - rev 307531
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Move image name verification to Python; r=dustin MozReview-Commit-ID: 8KJZH5vjANS
95aeafbfec097edaa2ce746811cb1d1de63a7d40: Bug 1290531 - Move docker validation from build.sh to Python; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 12:58:39 -0700 - rev 307530
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Move docker validation from build.sh to Python; r=dustin Now that we have a mach command and Python code for doing Docker image building, we can start moving code from build.sh to Python. We start with searching for and validating the `docker` binary works. MozReview-Commit-ID: 2DCc3b8UyZ3
4509921b0cfd298f27e5164d8ee6361fb2ae2eb2: Bug 1290531 - Remove unused find_registry(); r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 12:55:48 -0700 - rev 307529
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Remove unused find_registry(); r=dustin We're about to make significant changes to this file. Nuke an unused function to make diffs easier to reason about. MozReview-Commit-ID: KXXSrCVHww1
b6eff3784cf2bc9583c5c8476816e4ade138fb46: Bug 1290531 - Add mach taskcluster-build-image command; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 12:45:25 -0700 - rev 307528
Push 30969 by gszorc@mozilla.com at Mon, 01 Aug 2016 18:28:55 +0000
Bug 1290531 - Add mach taskcluster-build-image command; r=dustin Docker image building will soon need to use Python in order to produce the image build context archive. As the first step towards this, we introduce a Python function that calls out to build.sh. We also implement a mach command that calls it so we can test the functionality. I'm not keen about introducing a new mach command. I'd prefer to have a sub-command instead. I'm not sure what all uses `mach taskcluster-load-image`. Perhaps we could make a `taskcluster` top-level command. Or perhaps we could fold these image commands into `mach taskgraph`? Either way, the mach side of this isn't terribly important to the commit series: most of the code will live inside a Python module outside of mach. MozReview-Commit-ID: AI8p6H4psNh
d645b6c90e9dd518524629edce8b9efbafcba810: Bug 1290602: add TransformTask to task-graph machinery; r=gps
Dustin J. Mitchell <dustin@mozilla.com> - Mon, 01 Aug 2016 17:52:52 +0000 - rev 307527
Push 30968 by dmitchell@mozilla.com at Mon, 01 Aug 2016 18:04:23 +0000
Bug 1290602: add TransformTask to task-graph machinery; r=gps This abstracts out the process of running transforms as specified in kind.yml, with some useful default behavior that can be overridden by subclasses. MozReview-Commit-ID: 8pY4IYaN7NU
9570fe2c91f7861eede584e5e6f0a99d54c3b021: Backed out changeset 4c8d8454a4c5 (bug 1290931) because dependencies of this landed to inbound instead of autoland, causing future merge conflicts a=backout
Wes Kocher <wkocher@mozilla.com> - Mon, 01 Aug 2016 10:06:38 -0700 - rev 307526
Push 30967 by kwierso@gmail.com at Mon, 01 Aug 2016 17:06:48 +0000
Backed out changeset 4c8d8454a4c5 (bug 1290931) because dependencies of this landed to inbound instead of autoland, causing future merge conflicts a=backout
ff654490bc1ec85ccf23b3e3687a15f788b9921a: Bug 1290617 - allow non-temp mock webextensions r=kmag
Robert Helmer <rhelmer@mozilla.com> - Fri, 29 Jul 2016 16:14:19 -0700 - rev 307525
Push 30966 by rhelmer@mozilla.com at Mon, 01 Aug 2016 16:59:06 +0000
Bug 1290617 - allow non-temp mock webextensions r=kmag MozReview-Commit-ID: JeEeMkrQ4te
4c8d8454a4c54dab9f20541906d97d72ef61ce86: Bug 1290931: Don't use the exponential operator in tests; r=till
Benjamin Bouvier <benj@benj.me> - Mon, 01 Aug 2016 18:48:20 +0200 - rev 307524
Push 30965 by bbouvier@mozilla.com at Mon, 01 Aug 2016 16:56:54 +0000
Bug 1290931: Don't use the exponential operator in tests; r=till MozReview-Commit-ID: HTUmcNj7SEZ
6ea38e6856796ace377833cffd28dfc06dc362e8: Bug 1289643 - Allow level-{{level}}-checkouts cache to be used on Try; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 11:10:48 -0700 - rev 307523
Push 30964 by gszorc@mozilla.com at Mon, 01 Aug 2016 16:55:46 +0000
Bug 1289643 - Allow level-{{level}}-checkouts cache to be used on Try; r=dustin Previously, the VCS checkout cache would be optimized away on Try. This meant every task on Try had to create its own checkout from scratch. This meant writing ~150,000 files. By allowing use of the cache, we minimize the amount of work tasks need to perform on Try and therefore make tasks complete faster, on average. MozReview-Commit-ID: EWuic8lFVtv
50b87c38d18dc57a57de8b5c24079899180af11a: Bug 1289643 - Change path for checkouts from "workspace" to "checkouts"; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 10:44:19 -0700 - rev 307522
Push 30964 by gszorc@mozilla.com at Mon, 01 Aug 2016 16:55:46 +0000
Bug 1289643 - Change path for checkouts from "workspace" to "checkouts"; r=dustin Currently, TaskCluster tasks tend to use the "workspace" directory as a cache that manages the source checkout *and* additional state. Historically at Mozilla, we've lumped "source checkout" and "workspace" (sometimes known as an "objdir") into the same directory. This is not ideal. Ideally, there is an immutable, read-only source checkout and all files produced from that source live in a separate directory. In this commit, the "workspace" directory for the "lint" image has been renamed to "checkouts" and all tasks using the image have been updated accordingly. By having "checkout" in the name, we clearly identify this cache as being relevant to source checkouts, which IMO can serve a different role from "workspaces." This distinction is important, as the next commit will prevent the "checkouts" cache from getting optimized out in certain tasks. To hammer this point home, documentation on common caches has been introduced. MozReview-Commit-ID: BSEc4dM5YCt
0beb6afa58b70382dc1974bd07326bb8b948561a: Bug 1289643 - Add $ to regular expression in hg-shared cache name; r=dustin
Gregory Szorc <gps@mozilla.com> - Fri, 29 Jul 2016 10:29:25 -0700 - rev 307521
Push 30964 by gszorc@mozilla.com at Mon, 01 Aug 2016 16:55:46 +0000
Bug 1289643 - Add $ to regular expression in hg-shared cache name; r=dustin All the other expressions are anchored via ^ and $. MozReview-Commit-ID: BkCbZZUnwK6
eb9e8982f50cabcf45c596fb1fd9a2c97baad462: Backed out changeset 11b27019a256 (bug 1290710) for PGO build bustage in media/ffvpx/libavcodec/target a=backout
Wes Kocher <wkocher@mozilla.com> - Mon, 01 Aug 2016 09:43:07 -0700 - rev 307520
Push 30963 by kwierso@gmail.com at Mon, 01 Aug 2016 16:43:20 +0000
Backed out changeset 11b27019a256 (bug 1290710) for PGO build bustage in media/ffvpx/libavcodec/target a=backout
192c7f9edb1cfbb7d6cfdd9e526ac0a766a9156b: Bug 1290505 - Remove E10S NPAPI plugin windows special case. r=jimm
Benoit Girard <b56girard@gmail.com> - Fri, 29 Jul 2016 16:39:50 -0400 - rev 307519
Push 30962 by b56girard@gmail.com at Mon, 01 Aug 2016 14:18:10 +0000
Bug 1290505 - Remove E10S NPAPI plugin windows special case. r=jimm MozReview-Commit-ID: 7F7JeVmudjq
738847eb38313539c6efc7678a53d84ee9fb0266: Bug 1290813 - Include <ol reversed> reftests in wpt manifest, add test for nested div; r=xidorn
Manish Goregaokar <manishearth@gmail.com> - Mon, 01 Aug 2016 16:41:40 +0530 - rev 307518
Push 30961 by xquan@mozilla.com at Mon, 01 Aug 2016 11:21:52 +0000
Bug 1290813 - Include <ol reversed> reftests in wpt manifest, add test for nested div; r=xidorn MozReview-Commit-ID: HtyLrSiX0KS
(0) -300000 -100000 -30000 -10000 -3000 -1000 -300 -100 -50 -20 +20 +50 +100 +300 +1000 +3000 +10000 +30000 +100000 tip