3d3f055ade5269d45ae3acc45c91da8390c25554: Bug 1207772: Add some better sanity checks for the bootstrap function calls. r=rhelmer
Dave Townsend <dtownsend@oxymoronical.com> - Tue, 15 Sep 2015 10:45:52 -0700 - rev 297659
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1207772: Add some better sanity checks for the bootstrap function calls. r=rhelmer Unifies the methods we have to check that bootstrap add-ons are correctly loaded and makes it easier to make changes to them all in the future without needing to re-sign add-ons etc. This code allows a bootstrap script to use a shared script in a single line of code. The shared scripts sends out all the relevant info over the observer service, the add-ons manager test harness receives and retains the current state for every add-on also performing sanity checks like making sure an "install" method is always called before any "startup" method etc. It also provides simple functions to check the state of a given add-on.
1264e01362f3c5b49899fe826038029b125782e3: No bug. Fix Loop's eslint configuration, due to fallout from bug 1203520. rs=dmose NPOTB DONTBUILD
Mark Banner <standard8@mozilla.com> - Wed, 30 Sep 2015 18:50:05 +0100 - rev 297658
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
No bug. Fix Loop's eslint configuration, due to fallout from bug 1203520. rs=dmose NPOTB DONTBUILD
290304bf822b3dea5706a13c149d1a27da43fa6f: Bug 1209898 Fix for querying all history items older than 6 months r=sebastian
vivek <vivekb.balakrishnan@gmail.com> - Wed, 30 Sep 2015 19:09:05 +0300 - rev 297657
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1209898 Fix for querying all history items older than 6 months r=sebastian
7221fb39f8e5c620a4294c35a093864e10911ae4: Bug 1204158: Skip downloading system add-ons when we already have a copy locally. r=rhelmer
Dave Townsend <dtownsend@oxymoronical.com> - Thu, 24 Sep 2015 14:55:27 -0700 - rev 297656
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1204158: Skip downloading system add-ons when we already have a copy locally. r=rhelmer When we have a local copy of an add-on in the updated add-on set try to make a temporary copy rather than re-downloading.
addd92d9e21b06d7e1a0ec4663d15129d6eb2a99: Bug 1204159: Switch to existing system add-ons when they match the wanted set. r=rhelmer
Dave Townsend <dtownsend@oxymoronical.com> - Thu, 24 Sep 2015 14:26:13 -0700 - rev 297655
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1204159: Switch to existing system add-ons when they match the wanted set. r=rhelmer Checks the set that balrog gives us against the existing sets and switches without needing to download again.
53fbb6d6f8d3d1910ba3557795ef95b42af5f500: Bug 1193674 - If room context/name is unavailable, the title of the standalone is displayed as "{{roomName}} - Firefox Hello". r=Standard8
Manuel Casas <manuel.casasbarrado@gmail.com> - Wed, 30 Sep 2015 16:37:31 +0100 - rev 297654
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1193674 - If room context/name is unavailable, the title of the standalone is displayed as "{{roomName}} - Firefox Hello". r=Standard8
9d83fd4e40db9ab9e09610e68575be85ddee4c93: Bug 1209029 - .settings-menu .icon classes in Loop's panel are obsolete and should be removed. r=Standard8
Mark Banner <standard8@mozilla.com> - Wed, 30 Sep 2015 16:37:31 +0100 - rev 297653
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1209029 - .settings-menu .icon classes in Loop's panel are obsolete and should be removed. r=Standard8
4fd815c46a675085b7bb23fdd1cd91bf875cda19: Bug 1203520 - We need a DevTools ESLint plugin containing our ruleset r=gps
Michael Ratcliffe <mratcliffe@mozilla.com> - Wed, 30 Sep 2015 14:44:48 +0100 - rev 297652
Push 5980 by dtownsend@mozilla.com at Wed, 30 Sep 2015 19:15:49 +0000
Bug 1203520 - We need a DevTools ESLint plugin containing our ruleset r=gps
3805d38e6c2cd2c7f50f8a67b687dd14cb4a2578: Bug 1111689: Part 2 - Update ext-shader-texture-lod WebGL conformance test,r=jgilbert draft
Kearwood (Kip) Gilbert <kgilbert@mozilla.com> - Tue, 08 Sep 2015 15:27:46 -0700 - rev 297651
Push 5979 by kgilbert@mozilla.com at Wed, 30 Sep 2015 18:50:55 +0000
Bug 1111689: Part 2 - Update ext-shader-texture-lod WebGL conformance test,r=jgilbert - Cherry picked minimal changes to support updated ext-shader-texture-lod webgl conformance test.
3e6a254e65df944d3fd7f833d37616636fce2619: Bug 1111689 - Part 1: Let EXT_shader_texture_lod play when ARB_shader_texture_lod can play. r=jgilbert draft
Milan Sreckovic <milan@mozilla.com> - Mon, 09 Feb 2015 17:53:38 +0800 - rev 297650
Push 5979 by kgilbert@mozilla.com at Wed, 30 Sep 2015 18:50:55 +0000
Bug 1111689 - Part 1: Let EXT_shader_texture_lod play when ARB_shader_texture_lod can play. r=jgilbert
b6a062de70ddd3ea6030ea70854df147e807b76f: Bug 1210074 - Testcase. draft
Mike Conley <mconley@mozilla.com> - Wed, 30 Sep 2015 14:40:37 -0400 - rev 297649
Push 5978 by mconley@mozilla.com at Wed, 30 Sep 2015 18:41:36 +0000
Bug 1210074 - Testcase.
00a89e3c7698ebf2c7dbd85bb3576019fa15ffae: Build fix draft
Mike Conley <mconley@mozilla.com> - Wed, 30 Sep 2015 13:10:19 -0400 - rev 297648
Push 5978 by mconley@mozilla.com at Wed, 30 Sep 2015 18:41:36 +0000
Build fix
7362d4a071bf6649f42e997d4f0fba5129db5ca6: Bug 1208320 - Decrease compression level of test zip archives; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:33:04 -0700 - rev 297647
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Decrease compression level of test zip archives; r=glandium Compressing C++ unit tests is a long pole when writing test archives. Experimenting with various levels of compression revealed that compression level 9 was providing minimal space savings for significantly longer archiving times and greater CPU usage. Results of our experimentation of `make -sj8 package-tests` on OS X with various levels of compression are below. Note: these numbers were accidentally obtained without JS tests being archived. This skews the results a little but doesn't impact the analysis below. ARCHIVE SIZE WALL CPU (L=9) cppunittest 76,806,629 30.6s mochitest 61,276,928 9.4s reftest 31,204,396 11.0s ALL 228,146,761 31.2s 75.9s (L=8) cppunittest 76,851,593 24.1s mochitest 61,279,322 8.9s reftest 31,207,867 10.4s ALL 228,228,096 24.9s 64.7s (L=7) cppunittest 77,102,292 14.3s mochitest 61,305,147 8.2s reftest 31,260,359 9.4s ALL 228,717,803 15.0s 49.1s (L=6) cppunittest 77,321,408 11.5s mochitest 61,336,539 8.2s reftest 31,303,604 9.2s ALL 229,123,307 12.2s 44.7s (L=5) cppunittest 78,226,404 8.2s mochitest 61,483,804 7.6s reftest 31,509,349 8.8s ALL 230,725,600 9.6s 39.7s (L=4) cppunittest 79,733,669 6.3s mochitest 61,825,519 7.6s reftest 31,924,171 8.4s ALL 233,669,991 9.0s 36.4s (L=3) cppunittest 82,380,731 5.8s mochitest 62,554,431 7.1s reftest 32,696,415 8.1s ALL 239,180,168 8.9s 34.6s Levels lower than 3 resulted in larger archives with no decreae in wall time and marginal decrease in CPU time. As we can see, lowering the compression level reduces archiving time by >3x while only increasing total archive size by ~2.5 MB or ~1% for compression level 5. Total time hits a plateau around levels 4 and 5. After that, file size increases faster for little decrease in wall time. I suspect that we're hitting Python limits from having to process thousands of files: there's only so fast Python can do I/O and make function calls. I think choosing 4 or 5 for the new compression level are acceptable. I went with 5 because the wall time savings from 5 to 4 are marginal and the archive size does start to increase a bit faster at 4. That being said, 4 does consume 10% less CPU. I could easily just 4 as well. 5 is more conservative. We can always change to 4 after seeing results in the wild. The end result of this change is `make package-tests` is much faster: Before: 228,146,761 bytes; 31.2s wall; 75.9s CPU After: 230,725,600 bytes; 11.4s wall; 45.0s CPU Delta: +2,578,839 bytes; -19.8s wall; -30.9s CPU When you take the whole series into consideration: Before: 44.2s wall; 84.6s CPU After: 11.4s wall; 45.0s CPU Lowering CPU is impressive considering we switched from the C `zip` implementation to Python! Keep in mind we were at ~78s wall before e87b74b3db43 introduced concurrent archive generation! And we still haven't eliminated the staging of JS tests, which are several thousand files and a few dozen MB!
e9d74c160f4320d2e359eaed0cf410eaa615b073: Bug 1208320 - Support configuring zlib compression level; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:31:00 -0700 - rev 297646
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Support configuring zlib compression level; r=glandium An upcoming commit will introduce a caller that doesn't want the maximum compression level. This commit introduces arguments to control the compression level inside written archives.
78a6147c6d71bde5b331add4effdba39401deab3: Bug 1208320 - Print message when done with archiving; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:29:22 -0700 - rev 297645
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Print message when done with archiving; r=glandium Metrics are nice. Adding this output clearly demonstrates that C++ unit tests are the long pole by far: they take ~95% of wall execution time to archive (~30s total). The next longest archive only takes ~11s to produce. This will be important if we ever want to reduce archive time further on optimal hardware. FWIW, disabling compression will produce a C++ unit test archive in 1.0s. Archives with more files take longer, despite the significantly smaller sizes.
4468fd1eefb26b7c4cccafa4a3aa91259b5a2268: Bug 1208320 - Do not stage some C++ unit test support files before archiving; r?glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:27:54 -0700 - rev 297644
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Do not stage some C++ unit test support files before archiving; r?glandium Won't impact performance much. But fewer make foo makes porting the C++ unit tests (which are the largest remaining tests) to the Python archiver easier to grok. This conversion did change behavior slightly. Previously, startup cache files weren't being packaged if startup cache was disabled. Now, we always package them since their presence in the test archive should be harmless. The original change to guard their inclusion in ee82e0ae5488 was probably unnecessary.
16cd854e7a68df2e5aadbb945e833174a5d4c714: Bug 1208320 - Do not stage mozbase files before archiving; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:15:51 -0700 - rev 297643
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Do not stage mozbase files before archiving; r=glandium This prevents copying of 447 files adding to ~4 MB.
5815505520ae6d20dbd2bce016df69d2a7513c0e: Bug 1208320 - Do not stage JS test modules before archiving; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:04:58 -0700 - rev 297642
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Do not stage JS test modules before archiving; r=glandium Saves 400 KB over 40 files on my machine.
4d591efaee5a9d19aa40ec08fdb7eab6c2496e6c: Bug 1208320 - Do not stage TPS files before archiving; r=glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:03:32 -0700 - rev 297641
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Do not stage TPS files before archiving; r=glandium This saves copying of ~100 files comprising ~1 MB. Not significant. But it gets us a little closer to no staging.
2cb39f693ca9120ecfe6aab0aac3d82f5dd35c7c: Bug 1208320 - Do not stage reftest test files before archiving; r?glandium draft
Gregory Szorc <gps@mozilla.com> - Wed, 30 Sep 2015 11:02:20 -0700 - rev 297640
Push 5977 by gszorc@mozilla.com at Wed, 30 Sep 2015 18:40:40 +0000
Bug 1208320 - Do not stage reftest test files before archiving; r?glandium This is slightly more involved than earlier changes because reftests have a one-off mechanism for finding files. Essentially, the master reftest manifest is loaded, directories are discovered, and every file in those directories is packaged. We add support to our test archive generation tool to read sources from reftest manifests and tell it where the reftest manifests are. print-manifest-dirs.py was only being used for staging reftest files. Since we don't do that any more, the functionality doesn't need to exist in a standalone file, so it has been moved inline into test_archive.py. This change avoids copying ~26,000 tests consuming 131 MB during test packaging. This is a majority of the file count that was remaining in the stage directory at this point. On my machine (which hasn't typically seen major wall time wins from not staging files due to its fast SSD), this change made test packaging ~20% faster, reducing wall time from ~50s to ~40s! A Try push seemed to indicate drastic results with the series up to this point. Including the already landed changes to generate test archives concurrently, test packaging times on OS X builders dropped from ~18:40 to 6:29! Times on Linux x64 remained about the same (~2:46). This is possibly due to these machines already having SSDs and due to normal variance in performance of builders and EC2 instances.
(0) -100000 -30000 -10000 -3000 -1000 -300 -100 -50 -20 +20 +50 +100 +300 +1000 +3000 +10000 +30000 +100000 +300000 tip