Bug 1405408 - Pre: Handle query string and fragments in URL parameters. r=dustin
authorNick Alexander <nalexander@mozilla.com>
Thu, 05 Oct 2017 13:30:21 -0700
changeset 385702 831c287a4ae882823ec7411a00b05e88e69ebda2
parent 385701 911bafc2b6858b1ef0c2b98c3e1f9ed714caa41b
child 385703 347192dde0ed068e46593b68d465c6d1e4cd8db7
push id32664
push userarchaeopteryx@coole-files.de
push dateThu, 12 Oct 2017 09:34:55 +0000
treeherdermozilla-central@a32c32d9631c [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersdustin
bugs1405408
milestone58.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1405408 - Pre: Handle query string and fragments in URL parameters. r=dustin This was simply oversight before. I ran into this using the taskcluster-proxy /bewit interface, which returns a URL of the form https://domain.net/short/path/to.file?bewit="several thousand characters", which leads to an IOError due to the long path. Let's assume that such query strings and fragments are transient; we should drop these parts of the fetched URLs when writing to disk. MozReview-Commit-ID: FMJHMp7a3rA
python/mozbuild/mozbuild/artifacts.py
--- a/python/mozbuild/mozbuild/artifacts.py
+++ b/python/mozbuild/mozbuild/artifacts.py
@@ -785,17 +785,19 @@ class ArtifactCache(object):
             binascii.unhexlify(fname)
         except TypeError:
             # We download to a temporary name like HASH[:16]-basename to
             # differentiate among URLs with the same basenames.  We used to then
             # extract the build ID from the downloaded artifact and use it to make a
             # human readable unique name, but extracting build IDs is time consuming
             # (especially on Mac OS X, where we must mount a large DMG file).
             hash = hashlib.sha256(url).hexdigest()[:16]
-            fname = hash + '-' + os.path.basename(url)
+            # Strip query string and fragments.
+            basename = os.path.basename(urlparse.urlparse(url).path)
+            fname = hash + '-' + basename
 
         path = os.path.abspath(mozpath.join(self._cache_dir, fname))
         if self._skip_cache and os.path.exists(path):
             self.log(logging.DEBUG, 'artifact',
                 {'path': path},
                 'Skipping cache: removing cached downloaded artifact {path}')
             os.remove(path)