Bug 1484012: [fetch-content] Transparently decompress artifacts; r=gps
authorTom Prince <mozilla@hocat.ca>
Wed, 15 Aug 2018 15:53:27 -0600
changeset 491138 bd9d1bad72b6afe8e7aa40d7e0076c41357fd20f
parent 491137 99ba9748f54925885e6be457090a61019a75e636
child 491139 fa7466bb8cf83ad1b935bed362c2a46dcc8586b3
push id1815
push userffxbld-merge
push dateMon, 15 Oct 2018 10:40:45 +0000
treeherdermozilla-release@18d4c09e9378 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersgps
bugs1484012
milestone63.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1484012: [fetch-content] Transparently decompress artifacts; r=gps generic-worker transparently compresses uncompressed artifacts. Teach fetch-content to decompress those artifacts. Differential Revision: https://phabricator.services.mozilla.com/D3555
taskcluster/scripts/misc/fetch-content
--- a/taskcluster/scripts/misc/fetch-content
+++ b/taskcluster/scripts/misc/fetch-content
@@ -55,16 +55,19 @@ def stream_download(url, sha256=None, si
     """
     log('Downloading %s' % url)
 
     h = hashlib.sha256()
     length = 0
 
     t0 = time.time()
     with urllib.request.urlopen(url) as fh:
+        if not url.endswith('.gz') and fh.info().get('Content-Encoding') == 'gzip':
+            fh = gzip.GzipFile(fileobj=fh)
+
         while True:
             chunk = fh.read(65536)
             if not chunk:
                 break
 
             h.update(chunk)
             length += len(chunk)