Bug 1260241 - Implement Fennec packaging in Python. r=glandium A few notes: * This doesn't accommodate general OMNIJAR_NAME definitions. The current name (assets/omni.ja) is baked into the product in a few places, and is very unlikely to change, so we just error out if this isn't true. * This makes the package-manifest.in file authoritative for what goes into assets/, libs/, and the APK root. Previously, package-manifest.in wrote into assets/ and libs/ but upload-files-APK.mk also had a convoluted DIST_FILES filtering process to work through before a file actually made it into the APK. * This is intentional about repackaging. It simplifies the repackage step rather than trying to make unpackage-then-repackage the same as just package. I pretty much never get repackaging correct the first time; this should help. (I've manually tested it.) * The ALREADY_SZIPPED during repackaging is subsumed by the previous check if UNPACKAGE is set. The custom linker expects stored, not deflated, libraries, so there's some small legwork to accommodate that in mozjar. MozReview-Commit-ID: JvVtIUSX685

Bug 1260241 - Pre: Add compress option to Jarrer.add. r=glandium MozReview-Commit-ID: FWXeBOi9FTY

Bug 1249849: Ensure that we can always reach malloc_decl.h even when MOZ_REPLACE_ALLOC is not defined; r=glandium MozReview-Commit-ID: 5ok6K0cLJrt

Bug 1260402 - Disable PulseAudio only on Gonk for B2G r=glandium MozReview-Commit-ID: 8duWq1yDiJW

Bug 1260427 - Additional nspr/nss related old_configure_options. r=glandium MozReview-Commit-ID: Bre9H6hPlz5

Bug 1181142 - Part 2: Make the minimum jemalloc4 allocation size 16 bytes on Windows 64. r=glandium

Bug 1181142 - Part 1: Make the minimum allocation size word sized on all platforms. r=glandium Bug 691003 made the minimum allocation size word sized for Linux and OS X, we now need to do a similar change on Windows as well. For Windows the requirement is 8-bytes on 32-bit and 16-bytes on 64-bit.

Bug 1257870 - Require 1.5.0 --with-system-libvpx. r=glandium WebRTC code requires VP9E_SET_NOISE_SENSITIVITY from vpx/vp8cx.h which was added in 1.4. We could feature test, but gating on the minimum version seems better given the m4 code is to be replaced by python. We also depend on VPX_MAX_LAYERS wich is part of the 1.5.0 release. This dependency was added with an SVC support patch in May 2015, bug 1237023. Optional --enable-spatial-svc is handled by the work-around in bug 1248335. MozReview-Commit-ID: 79kg9JSFtJM

Bug 1259772 - don't pass -fno-reorder-functions to clang on Android; r=glandium

bug 1247396 - drop support for MOZ_SHARED_ICU. r=glandium MozReview-Commit-ID: 54AQdzJqvvA

Bug 1181142 - Part 1: Make the minimum allocation size word sized on all platforms. r=glandium Bug 691003 made the minimum allocation size word sized for Linux and OS X, we now need to do a similar change on Windows as well. For Windows the requirement is 8-bytes on 32-bit and 16-bytes on 64-bit.

Bug 1239789 - Add secondary buffers to zxx_stream::StaticAllocator to handle reentrancy. r=glandium

Bug 1258257 - Reduce mozjemalloc page cache size from 4 MiB to 1 MiB. r=glandium. This reduces memory usage by up to 3 MiB per process. MozReview-Commit-ID: Gfs9PIJM4br

Bug 1249849: Make sure the correct implementations of malloc and free are used in DLL blocklist; r=glandium MozReview-Commit-ID: 7piMI6MwOZh

Bug 1258175 - Remove broken -Wunreachable-code-return and -Wunreachable-code-aggressive checks. r=glandium compiler-opts.m4's check for `MOZ_CXX_SUPPORTS_WARNING(-W, unreachable-code-return, ac_cxx_has_wunreachable_code_return)` is broken. The C/C++ code that configure emits for MOZ_CXX_SUPPORTS_WARNING() actually contains a -Wunreachable-code-return warning and, thus, doesn't actually detect that clang supports the -Wunreachable-code-return flag. This configure code in MOZ_CXX_SUPPORTS_WARNING(): AC_TRY_COMPILE([], [return(0);], $3="yes", $3="no") generates something like: int main() { return(0); ; return 0; } where the second return, automatically emitted by configure, is unreachable and causes a -Wunreachable-code-return warning. The fix is to remove the redundant return(0) from MOZ_CXX_SUPPORTS_WARNING(). This allows clang's -Wunreachable-code-return flag to be detected, but then -Wunreachable-code-return breaks other configure checks, including third-party libraries' configure checks (in particular jemalloc) that also have redundant `return(0)`. So all the third-party libraries' configure checks would need to be fixed upstream, which seems like more hassle than the value of the -Wunreachable-code-return warnings.

Bug 1258294 - Fix typo in Mulet moz.configure referencing wrong configure file r=glandium MozReview-Commit-ID: 8tXlm29uyt

Bug 1256995 - Move --with-gradle to mobile/android/moz.configure. r=glandium This also adds a GRADLE_FLAGS environment variable for use in automation. Manually tested. MozReview-Commit-ID: 8nDkqz2VnJn

Bug 1258574 - Allow JARReader to read data already in memory; r=glandium This will be needed to teach artifact builds to extract files from omni.ja files whose content is loaded into memory (from a tar archive). MozReview-Commit-ID: LH2HkKx5Zj3

Bug 1256694 - Loop should use MOZ_APP_MAXVERSION to generate the maximum supported Firefox version number to correctly support security builds (build sytem part). r=glandium

Bug 1257435 - Package Universal CRT DLLs with jsshell; r=glandium Builds with Visual Studio 2015 require the Universal CRT. While the universal CRT may be present on the target machine, there is no guarantee of this. So, we have to package these DLLs to ensure target machines are able to run js.exe. Similar code to what this commit adds exists in build/win32/Makefile.in. MozReview-Commit-ID: 8LIk1JlKLiT

Bug 1257435 - Move MSVC runtime DLL code outside of MOZ_SYSTEM_NSPR block; r=glandium Why it was the old way, I'm not sure. MozReview-Commit-ID: FMmDbZI0NqL

Bug 1257435 - Reformat make syntax to make easier to read; r=glandium MozReview-Commit-ID: Ch86LX8Fx0q

Bug 1256966 - Move NIGHTLY_BUILD and RELEASE_BUILD to Python configure. r=glandium MozReview-Commit-ID: 2Z3XMO2n681

Bug 1257685 - Always read a mozconfig when printing help from python configure. r=glandium This restores help for project specific options. MozReview-Commit-ID: 3n0TBnUb0K7

bug 1257607 - Add Version type to moz.configure. r=glandium This change adds a `Version` type to moz.configure which is a small wrapper around `distutils.version.Version`. It's suitable for wrapping version numbers in configure checks and doing equality or greater-than less-than comparisons in a sensible way. MozReview-Commit-ID: BOL6yvemulG

bug 1257542 - check_prog template should sanity check that progs is not a string. r=glandium In Python parens around an expression without a trailing comma is not a tuple, so ('foo') == 'foo'. This is really easy to screw up with check_progs, which coerces progs to a list and would give you ['f','o','o'] in this case. This patch enforces that the progs argument is a tuple or list and errors if it is not. MozReview-Commit-ID: 7BJZuF9B8D5

Bug 1231320: support min_scm_level and default secret specs; r=garndt,glandium,mshal This improves support for fetching secrets to skip fetching, optionally replacing with a default value, for builds based on trees with SCM levels below a minimum. In practical terms, this allows try builds to operate without secrets. MozReview-Commit-ID: 5DYSfzkYZQv

Bug 1250709 - Upgrade browser to gcc 4.9.3, r=glandium

bug 1256614 - replace mozglue/linker/tests/Makefile.in with a PYTHON_UNIT_TEST. r=glandium MozReview-Commit-ID: 4qCDAKEY8b5

Bug 1092004 - Use getdtablesize for non-gonk builds as well. r=glandium When building non-gonk builds, ANDROID_VERSION is not set. Beginning with NDK 11, getdtablesize is no longer included. This means that we should use the stub version of the function that is defined in android_stub.h for all android platforms. This patch moves the function out of the "#if ANDROID_VERSION >=21" block so that all android code can use it. Adding glandium as the reviewer, because he reviewed the original patch at bug 1103816. MozReview-Commit-ID: 2NmUl5XuvDS

Bug 1257005 - Don't use debug runtime in standalone SpiderMonkey builds; r=glandium This patch effectively synchronizes code from old-configure.in to js/src/old-configure.in. Before, the top-level configure would set MOZ_NO_DEBUG_RTL when using MOZ_MEMORY on Windows. This would get inherited when the top-level configure invoked the js/src configure. Since js/src's configure didn't have the same code, we never set MOZ_NO_DEBUG_RTL when running a standalone js/src configure. This caused the debug runtime to be used in standalone SpiderMonkey builds. There is still some code in the root old-configure.in around CRT handling that js/src/old-configure.in doesn't have. This will likely get consolidated as part of the conversion to configure.py. MozReview-Commit-ID: CD834DhIlLS

Bug 1257005 - Don't use debug runtime in standalone SpiderMonkey builds; r=glandium This patch effectively synchronizes code from old-configure.in to js/src/old-configure.in. Before, the top-level configure would set MOZ_NO_DEBUG_RTL when using MOZ_MEMORY on Windows. This would get inherited when the top-level configure invoked the js/src configure. Since js/src's configure didn't have the same code, we never set MOZ_NO_DEBUG_RTL when running a standalone js/src configure. This caused the debug runtime to be used in standalone SpiderMonkey builds. MozReview-Commit-ID: CD834DhIlLS

Bug 1256033 - Remove layout/style/test/Makefile.in; r=glandium The css_properties.js rule can be converted into gen-css-properties.py, which we can install with TEST_HARNESS_FILES instead of the mochitest.ini manifest. MozReview-Commit-ID: F7nf71ORWsS

Bug 1256033 - Allow GENERATED_FILES scripts to depend on other generated files; r=glandium If we add an ObjDirPath as an input to a GENERATED_FILES script, we should run it in the misc tier to ensure the dependent files are created beforehand. We need to use Path objects instead of raw filenames in the GeneratedFile object so the recursive make backend can distinguish between source and objdir files. MozReview-Commit-ID: 9thHTi75zdI

Bug 1253436 - Write out a machine readable file with binaries metadata; r=glandium This will make it easier for binaries only archive generation to consult the list of binaries that are relevant to packaging. This does overlap somewhat with compile databases. Perhaps someday the logic could converge. While I was here, I cleaned up writing of the all-tests.json file to avoid an intermediate string variable. This patch adds to_dict methods on some frontend data types. There is room to improve the serialization of these types. For example, we could use __slots__ to drive the default formatter. We could also leverage a custom JSONEncoder class that knows how to call to_dict() on instances. In the spirit of perfect is the enemy of done, I think this should be deferred to a follow-up bug. MozReview-Commit-ID: XVnKB1MNqu

Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r=glandium Now when we print instances in a debugger, we'll see something like: <StaticLibrary: other-licenses/snappy/libother-licenses_snappy.a> <StaticLibrary: toolkit/library/StaticXULComponentsEnd/libStaticXULComponentsEnd.a> <SharedLibrary: toolkit/library/XUL> <StaticLibrary: toolkit/library/libxul_s.a> <HostProgram: config/nsinstall_real> <Program: memory/replace/logalloc/replay/logalloc-replay> <SimpleProgram: mfbt/tests/TestArrayUtils> ... instead of the the class name and memory address. MozReview-Commit-ID: 8zdrM6KfP8U

Bug 1253436 - Remove unused imports; r=glandium MozReview-Commit-ID: 494hmTwyldH

Bug 1250709 - Upgrade browser to gcc 4.9.3, r=glandium

Bug 1253436 - Write out a machine readable file with binaries metadata; r=glandium This will make it easier for binaries only archive generation to consult the list of binaries that are relevant to packaging. This does overlap somewhat with compile databases. Perhaps someday the logic could converge. While I was here, I cleaned up writing of the all-tests.json file to avoid an intermediate string variable. This patch adds to_dict methods on some frontend data types. There is room to improve the serialization of these types. For example, we could use __slots__ to drive the default formatter. We could also leverage a custom JSONEncoder class that knows how to call to_dict() on instances. In the spirit of perfect is the enemy of done, I think this should be deferred to a follow-up bug. MozReview-Commit-ID: XVnKB1MNqu

Bug 1253436 - Add __repr__ to BaseLibrary and BaseProgram; r=glandium Now when we print instances in a debugger, we'll see something like: <StaticLibrary: other-licenses/snappy/libother-licenses_snappy.a> <StaticLibrary: toolkit/library/StaticXULComponentsEnd/libStaticXULComponentsEnd.a> <SharedLibrary: toolkit/library/XUL> <StaticLibrary: toolkit/library/libxul_s.a> <HostProgram: config/nsinstall_real> <Program: memory/replace/logalloc/replay/logalloc-replay> <SimpleProgram: mfbt/tests/TestArrayUtils> ... instead of the the class name and memory address. MozReview-Commit-ID: 8zdrM6KfP8U

Bug 1253436 - Remove unused imports; r=glandium MozReview-Commit-ID: 494hmTwyldH

Bug 1255963 - Detect warnings without space after line number; r=glandium I'm not sure when this changed, but at least Visual Studio 2015 doesn't always emit a space between the line number and the ": warning" text in cl.exe output. Making the space optional in the regular expression enables one a VS2015 build to capture 375 warnings instead of 17. We still fail to capture some warnings (notably generic warnings about bad command arguments and linker warnings). But that can be dealt with later. MozReview-Commit-ID: q402CxTrQK

Bug 1254948 - Remove makefiles used to transition from jemalloc_config.c to jemalloc_config.cpp; r=glandium

Bug 1254987 - Get the subconfigure working properly for mail/configure.in. r=glandium

Bug 1254858: P9. Search libmozav* lib relative to lgpllibs. r=glandium XUL location can't be used as reference as its location varies. MozReview-Commit-ID: EKGVpjeepIR

Bug 1242632 - Remove TESTING_FILES from moz.build; r=glandium MozReview-Commit-ID: ALy4s6Xpg7f

Bug 1240149 - Build system changes necessary to run linux artifact builds in automation. r=glandium MozReview-Commit-ID: CJ0CTBtklCs

Bug 1254795 - Add missing --enable flags to old.configure. r=glandium

Bug 1252639 (part 2) - Move contents of plarenas.h into plarena.h. r=glandium. Because the plarenas.h/plarena.h is strange and confusing. But the patch keeps plarenas.h around in a way that the API doesn't change -- any code that included plarena.h or plarenas.h should not need changing.

Bug 1252639 (part 1) - Remove the shared PLArena freelist. r=glandium. It's not useful and just adds complexity.