Bug 1498715 - Use faster XZ compression preset for Android assets/ libraries for local developers. r=esawin
authorNick Alexander <nalexander@mozilla.com>
Mon, 15 Oct 2018 13:07:23 +0000
changeset 499757 f9cf33c52739b06905c182b286fa2b4d1b0f3890
parent 499756 92b4fac6ea6c74a9292a21fd086fa35f76c8ba92
child 499758 88d826bf6a68a5f9ead87b17f51ec3747b3e86a9
push id1864
push userffxbld-merge
push dateMon, 03 Dec 2018 15:51:40 +0000
treeherdermozilla-release@f040763d99ad [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersesawin
bugs1498715, 2, 31286204, 34877984
milestone64.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1498715 - Use faster XZ compression preset for Android assets/ libraries for local developers. r=esawin Before: xz-compressing dist/fennec/assets/x86/libxul.so with "xz -zkf dist/fennec/assets/x86/libxul.so --threads=1 --x86 --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0"... xz-compressing dist/fennec/assets/x86/libxul.so with "xz -zkf dist/fennec/assets/x86/libxul.so --threads=1 --x86 --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0"... DONE (69.13s) $ ls -al dist/fennec/assets/x86/libxul.so -rwxr-xr-x 1 nalexander staff 31286204 12 Oct 14:06 dist/fennec/assets/x86/libxul.so After: xz-compressing dist/fennec/assets/x86/libxul.so with "xz -zkf dist/fennec/assets/x86/libxul.so --threads=1 --x86 --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=fast,nice=273,mf=hc4,depth=24"... xz-compressing dist/fennec/assets/x86/libxul.so with "xz -zkf dist/fennec/assets/x86/libxul.so --threads=1 --x86 --lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=fast,nice=273,mf=hc4,depth=24"... DONE (19.52s) $ ls -al dist/fennec/assets/x86/libxul.so -rwxr-xr-x 1 nalexander staff 34877984 12 Oct 14:00 dist/fennec/assets/x86/libxul.so All timings on a late 2013 MacBook Pro (i.e., a very slow device). Differential Revision: https://phabricator.services.mozilla.com/D8614
python/mozbuild/mozpack/executables.py
--- a/python/mozbuild/mozpack/executables.py
+++ b/python/mozbuild/mozpack/executables.py
@@ -123,16 +123,18 @@ def elfhack(path):
     if subprocess.call(cmd) != 0:
         errors.fatal('Error executing ' + ' '.join(cmd))
 
 
 def xz_compress(path):
     '''
     Execute xz to compress the given path.
     '''
+    path = os.path.abspath(path)
+
     if open(path, 'rb').read(5)[1:] == '7zXZ':
         print('%s is already compressed' % path)
         return
     
     from buildconfig import substs
     xz = substs.get('XZ')
     cmd = [xz, '-zkf', path]
 
@@ -150,20 +152,33 @@ def xz_compress(path):
     elif substs.get('CPU_ARCH') == 'x86':
         bcj = '--x86'
 
     if bcj:
         cmd.extend([bcj])
 
     # We need to explicitly specify the LZMA filter chain to ensure consistent builds
     # across platforms. Note that the dict size must be less then 16MiB per the hardcoded
-    # value in mozglue/linker/XZStream.cpp. This is the default LZMA filter chain for for
-    # xz-utils version 5.0. See:
+    # value in mozglue/linker/XZStream.cpp. See:
     # https://github.com/xz-mirror/xz/blob/v5.0.0/src/liblzma/lzma/lzma_encoder_presets.c
     # https://github.com/xz-mirror/xz/blob/v5.0.0/src/liblzma/api/lzma/container.h#L31
-    cmd.extend(['--lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0'])
-    print('xz-compressing %s with %s' % (path, ' '.join(cmd)))
+
+    if not substs.get('DEVELOPER_OPTIONS'):
+        # This is the default LZMA filter chain for for xz-utils version 5.0. See:
+        cmd.extend(['--lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=normal,nice=64,mf=bt4,depth=0'])
+    else:
+        # This is the filter chain for level=3, which is the highest
+        # compression fast preset.
+        cmd.extend(['--lzma2=dict=8MiB,lc=3,lp=0,pb=2,mode=fast,nice=273,mf=hc4,depth=24'])
+
+    print('xz-compressing %s with "%s"...' % (path, ' '.join(cmd)))
+
+    import time
+    t0 = time.time()
 
     if subprocess.call(cmd) != 0:
         errors.fatal('Error executing ' + ' '.join(cmd))
         return
 
+    elapsed = time.time() - t0
+    print('xz-compressing %s with "%s"... DONE (%.2fs)' % (path, ' '.join(cmd), elapsed))
+
     os.rename(path + '.xz', path)