Bug 1312290 - Force package extension when building AUR packages on Arch Linux. r=gps Previously if an Arch Linux user had a different package extension configured in `/etc/makepkg.conf` building AUR packages during bootstrap would fail. Forcing the extension by providing it as an environment variable makes sure building doesn't fail regarding of a user's configuration. MozReview-Commit-ID: 4aryYS0XVr7

Bug 1313259 - Fix some preprocessor unit tests to actually do the tests they're supposed to do. r=gps

Bug 1312574 - Use cPickle instead of json to serialize the build system's tests database. r=gps MozReview-Commit-ID: 9xmUtVV6SRN

Bug 1303184 - Add exception list to find-dupes, and error out when unexpected dupes are detected; r=gps MozReview-Commit-ID: JRRnj6Kw1s4

Bug 1310841: Modify build rules to allow specification of a manifest for Windows DLLs; r=gps MozReview-Commit-ID: EdY8Kx8irYW

Bug 1312520 - Store and process manifest-level defaults in the build system separately from individual tests. r=gps MozReview-Commit-ID: 1dSMAaOqToJ

Bug 1309350: Part 2 - Speed up synchronous resolution of module paths. r=gps,ochameau r?gps for the build changes and ochameau for the rest. This results in about a 28% speed-up for Jetpack mochitest runs, for me. MozReview-Commit-ID: K30q7BfvTLs

Bug 1302776: document caches; r=gps MozReview-Commit-ID: Dy9aXGo5O4u

Bug 1305485 - use robustcheckout with ephemeral storage (tc windows); r=gps MozReview-Commit-ID: 1jatmsVJsDM

Bug 1305095 - Add a fallback hg fingerprint, supports timeouts in automation and supports local developers who wish to use 'run locally'. r=dustin,gps MozReview-Commit-ID: 66ctmZdSZkC

Bug 1296530 - Bustage followup part deux; r=gps

Bug 1296530 - Bustage followup; r=gps

Bug 1309657 - Update ccache stats parser for version 3.3. r=gps ccache 3.3 added two fields in the status. 1. "cache hit rate" https://github.com/ccache/ccache/issues/94 2. "cleanups performed" https://github.com/ccache/ccache/commit/0769ea3630efd6315303f7325fc16d73a0ef1a4d We just teach the parser the two fields. Nothing else. MozReview-Commit-ID: 9kZrDrpCopd

Bug 1305485 - use robustcheckout with ephemeral storage (tc windows); r=gps MozReview-Commit-ID: 1jatmsVJsDM

bug 1294565 - add a `mach vendor rust` command to vendor crates into third_party/rust. r=gps This command is mostly just a wrapper around `cargo vendor`, but it runs it the right way and will install the cargo-vendor tool if necessary. Additionally, the mach command will by default error if there are uncommited changes to files other than Cargo.{toml,lock} in your working copy, and it will stage changes to the vendored crates in your VCS after the update. MozReview-Commit-ID: 2fMDBawNUMO

bug 1294565 - add some more helpers to mozversioncontrol, add MozbuildObject.repository. r=gps I wanted to be able to do some VCS interaction from a mach command, and we didn't have anything suitable, so I tore up mozversioncontrol and replaced it with a framework to hang new features off of. I've only implemented the bits I need currently (get_modified_files and add_remove_files), but it should be straightforward to add more functionality there. This patch also adds a `repository` property to `MozbuildObject`, which will return a `Repository` object for the topsrcdir to make using these helpers even easier for `MozbuildObject`-derived classes. MozReview-Commit-ID: Gw6Ixp1ltiN

Bug 1304593 - Take universal binaries into account when detecting objdir mismatches, r=gps When attempting to run mach from a make target, there were failures on OSX due to an objdir mismatch: https://dxr.mozilla.org/mozilla-central/rev/7c576fe3279d87543f0a03b844eba7bc215e17f1/python/mozbuild/mozbuild/base.py#656 The two detected objdirs were "<objdir>" and "<objdir>/x86_64". I believe this should not count as a mismatch, otherwise it would not be possible to run a mach command from the x86_64 directory. MozReview-Commit-ID: CXDEABNWX8L

Bug 1304593 - Fix broken test_entry_point.py in mach, r=gps Fix and enable python/mach/mach/test/test_entry_poiny.py. MozReview-Commit-ID: FtMpt8Nmq3g

Bug 1305877 - Disable wpt navigation test causing hang later in the test run, r=gps MozReview-Commit-ID: 6PA6WMeQzRv

bug 1306662 - Make desktop-test and desktop1604-test pull minidump_stackwalk binary from the in-tree tooltool manifest. r=gps Instead of having the desktop[1604]-test images currently bake in a minidump_stackwalk binary from some random S3 URL, use the binary listed in the in-tree tooltool manifest that we use for buildbot test jobs. As a nice side-effect, this will ensure that the desktop-test images get rebuilt whenever that manifest is updated with a new version, so they will continue to use the right version in the future. MozReview-Commit-ID: 6bThddwq6p1

bug 1306662 - Make desktop-test and desktop1604-test pull minidump_stackwalk binary from the in-tree tooltool manifest. r=gps Instead of having the desktop[1604]-test images currently bake in a minidump_stackwalk binary from some random S3 URL, use the binary listed in the in-tree tooltool manifest that we use for buildbot test jobs. As a nice side-effect, this will ensure that the desktop-test images get rebuilt whenever that manifest is updated with a new version, so they will continue to use the right version in the future. MozReview-Commit-ID: 6bThddwq6p1

Bug 1305157 - Handle the build/ directory's GENERATED_FILES; r=gps This is a little tricky since tup expects rules in a single Tupfile to be topologically sorted, and the emitter emits the GeneratedFile object before it emits the FinalTargetPreprocessedFile object. We work around this by delaying when we process the application.ini.h GeneratedFile object, but it's a hack. MozReview-Commit-ID: 8nVm76nZOKb

Bug 1305157 - Handle FinalTargetPreprocessedFiles in the tup backend; r=gps MozReview-Commit-ID: 6fxTagN6mGl

Bug 1305157 - Fix buildid.h and create source-repo.h in make before tup runs; r=gps MozReview-Commit-ID: CPYA1OJUfo8

Bug 1305157 - Move GeneratedFile handling into its own function; r=gps MozReview-Commit-ID: HbZBYVugj9E

Bug 1304970 - Check encoding of js.msg-like files. r=jandem,gps

bug 1304042 - buildsymbols should fail if running dsymutil fails. r=gps MozReview-Commit-ID: LteC4jGF3FD

Bug 1305740: delete leftover legacy.py; r=gps MozReview-Commit-ID: 8GHv1E9CBRH

Bug 1304484 - Add back gecko vcs environment variables for desktop-test tasks, r=gps This was regressed by bug 1296397 which stopped setting the GECKO_HEAD_REPOSITORY and the like. It wasn't caught because the task doesn't actually depend on those environment variables except when using the interactive loaner. This patch supports vcs checkout regardless of whether the task runs from source or not. MozReview-Commit-ID: CDxdG8G6JDd

Bug 1304125 - Perform WebIDL code generation in the tup backend; r=gps MozReview-Commit-ID: Fnw8380zyGU

Bug 1304129 - Use xpidllex.py / xpidlyacc.py from GENERATED_FILES; r=gps MozReview-Commit-ID: 8w50kZWinzl

Bug 1304129 - Build GENERATED_FILES in the tup backend; r=gps MozReview-Commit-ID: IaVAuRWIjmC

Bug 1304131 - list xpidl.stub as the first output for header.py; r=gps Since xpidllex.py and xpidlyacc.py are created deep in the other-licenses/ply code, we can't easily pass in our FileAvoidWrite handle from the mozbuild action. As a result, when ply writes out xpidllex.py, the mozbuild action overwrites the contents of the file with a new 0-length xpidllex.py. This causes future xpidl invocations to write out a new xpidllex.py rather than using the pre-built one. MozReview-Commit-ID: NOC1Wr3MZO

Bug 1301334 - Package Marionette tests via moz.build entries r=gps Move packaging for Marionette from make to test_archiver by using root manifest files. MozReview-Commit-ID: 1cxEBYQeJ2Z **

Bug 1282522 - Ensure stderr is printed in case of a shutdown crash in a passing xpcshell test. r=gps There were two assumptions preventing this output from being logged, both related to the case a test passes and xpcshell returns 0. The first was that we would not find crash dumps in this case, and would therefore not need to log the full output of the test process (in the case xpcshell returned non-zero or a test failed, we would log this output prior to checking for crashes). The second was that if a test was eligible to retry, we wouldn't need to store a test's output at all, because this output would only relate to a failure that we would consider non-fatal. The first assumption does not hold because it's possible to fatally assert at shutdown in tests spawning child processes without causing a test failure or non-zero exit code. The second assumption followed from the first, and is violated when the first is violated, because in this case we would consider a found crash fatal even when a test was eligible to retry. This patch reverses these assumptions and logs the full output of a test that passes but produces crash dumps. It's not clear that the existing code intended for a crash to ever be considered fatal when a test was eligible to retry, but to change this criteria now would reduce our effective test coverage by ignoring crashes that are now considered fatal, so after this patch we continue to consider this scenario fatal. If it is determined these crashes are related to these tests running in parallel with other tests, or they are not relevant for some other reason, these tests should be run sequentially, or this criteria should be changed. MozReview-Commit-ID: 2PaFSGx2MVR

Bug 1302364 - Include firefox-ui and puppeteer in all-tests.json, and allow to run tests via "mach test". r=gps MozReview-Commit-ID: EwONsQSgAym

Bug 1302364 - Update TEST_MANIFESTS documentation for install_root in resolve_tests(). r=gps MozReview-Commit-ID: 9EXT3Jakwx1

Bug 1303210 - Show compressed size in find_dupes.py report, and sort its output. r=gps

Bug 1295542 - Ensure at least an empty chrome.manifest next to each omnijar file. r=gps

Bug 1292674 - Also allow android build-tools v23.0.1 in configure, r=gps I recently discovered that build-tools 23.0.3 doesn't seem to exist on Fedora. Although I fixed mozboot to download 23.0.1 instead, the builds were still failing because of configure required 23.0.3. This seems like an artificial limitation, as building with 23.0.1 seems to work just fine. This patch will allow either 23.0.1 or 23.0.3. Note: It would probably better to check for some "minimum" version of build-tools I think GNU sort has a -V option we could use, but I don't know how cross-platform this is. MozReview-Commit-ID: 8W0P3yyAHu1

Bug 1302364 - Include firefox-ui and puppeteer in all-tests.json, and allow to run tests via "mach test". r=gps MozReview-Commit-ID: EwONsQSgAym

Bug 1302364 - Update TEST_MANIFESTS documentation for install_root in resolve_tests(). r=gps MozReview-Commit-ID: 9EXT3Jakwx1

Bug 1300812 - Make Mozharness downloads handle better intermittent S3/EC2 issues r=aki,gps We believe we're getting incomplete bytes when fetching files from S3 to our EC2 instances. This patch should help give us more information and retry few times before failing. MozReview-Commit-ID: 7tUzZmS8Zph

Bug 1286075: fix flake8 error; r=gps CLOSED TREE a=KWierso MozReview-Commit-ID: 4DWMQTVekwk

Bug 1286075: remove trailing newline in env var; r=gps MozReview-Commit-ID: FebGZoVfVdj

Bug 1286075: add how-tos covering new functionality; r=gps MozReview-Commit-ID: 5l6v81UKskC

Bug 1286075: add a toolchain kind; r=gps MozReview-Commit-ID: EEeQdA9aPPq

Bug 1286075: add artifact-build kind; r=chmanchester r=gps MozReview-Commit-ID: 41WpDtXvoPb

Bug 1286075: allow defaults in TransformTask; r=gps MozReview-Commit-ID: 5tdnxykLTZc

Bug 1286075: add support for optimizing based on files changed in the push; r=gps MozReview-Commit-ID: 5di7TuL9X2P