servo: Merge #14452 - Commit that fixes the issue #11074 by upgrading pip whenever virtualeā€¦ (from PeterZhizhin:upgrade-pip-with-new-virtualenv); r=frewsxcv
authorPeter <piter.zh@gmail.com>
Sat, 03 Dec 2016 21:11:48 -0800
changeset 340273 ca239fcfa931c379dfc327af16e37358379f0fda
parent 340272 37bf28cc0b3321d0af8e509b8d8cdf28f53aa993
child 340274 825d00ebea71397dc0b4c0a4cd86015f386a3982
push id31307
push usergszorc@mozilla.com
push dateSat, 04 Feb 2017 00:59:06 +0000
treeherdermozilla-central@94079d43835f [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersfrewsxcv
servo: Merge #14452 - Commit that fixes the issue #11074 by upgrading pip whenever virtualeā€¦ (from PeterZhizhin:upgrade-pip-with-new-virtualenv); r=frewsxcv <!-- Please describe your changes on the following line: --> I have kind of resolved the issue #11074 by adding bool variable which is set to `True` if we had created the virtualenv and `False` otherwise. Then it updates pip by executing `pip install --upgrade pip` in the same way as packages are updated. I am a little bit worried that I have almost duplicated the installation routine from the `for` loop but I am not sure whether I should add a function or not. I think it is the best way of doing this because it does not need any Internet access for regular work (only for the first time you execute mach) as @larsbergstrom worried [here](https://github.com/servo/servo/pull/11149). It also doesn't add any extra latency on a no-op build. I have checked the solution inside a docker container based on debian wheezy. Before the patch `./mach` failed to run because it wasn't able to install some packages. Now it runs successfully. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [X] These changes fix #11074 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [X] These changes do not require tests because it changes only mach_bootstrap.py <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Source-Repo: https://github.com/servo/servo Source-Revision: ea59e7bb68e83ac2631dcdad24f270bff68092eb
servo/python/mach_bootstrap.py
--- a/servo/python/mach_bootstrap.py
+++ b/servo/python/mach_bootstrap.py
@@ -107,27 +107,30 @@ def _activate_virtualenv(topdir):
     virtualenv_path = os.path.join(topdir, "python", "_virtualenv")
     check_exec_path = lambda path: path.startswith(virtualenv_path)
     python = _get_exec_path(PYTHON_NAMES)   # If there was no python, mach wouldn't have run at all!
     if not python:
         sys.exit('Failed to find python executable for starting virtualenv.')
 
     script_dir = _get_virtualenv_script_dir()
     activate_path = os.path.join(virtualenv_path, script_dir, "activate_this.py")
+    need_pip_upgrade = False
     if not (os.path.exists(virtualenv_path) and os.path.exists(activate_path)):
         virtualenv = _get_exec_path(VIRTUALENV_NAMES)
         if not virtualenv:
             sys.exit("Python virtualenv is not installed. Please install it prior to running mach.")
 
         process = Popen([virtualenv, "-p", python, virtualenv_path], stdout=PIPE, stderr=PIPE)
         process.wait()
         if process.returncode:
             out, err = process.communicate()
             print('Python virtualenv failed to execute properly:')
             sys.exit('Output: %s\nError: %s' % (out, err))
+        # We want to upgrade pip when virtualenv created for the first time
+        need_pip_upgrade = True
 
     execfile(activate_path, dict(__file__=activate_path))
 
     python = _get_exec_path(PYTHON_NAMES, is_valid_path=check_exec_path)
     if not python:
         sys.exit("Python executable in virtualenv failed to activate.")
 
     # TODO: Right now, we iteratively install all the requirements by invoking
@@ -138,16 +141,30 @@ def _activate_virtualenv(topdir):
     # and it will check for conflicts.
     requirements_paths = [
         os.path.join("python", "requirements.txt"),
         os.path.join("tests", "wpt", "harness", "requirements.txt"),
         os.path.join("tests", "wpt", "harness", "requirements_firefox.txt"),
         os.path.join("tests", "wpt", "harness", "requirements_servo.txt"),
     ]
 
+    if need_pip_upgrade:
+        # Upgrade pip when virtualenv is created to fix the issue
+        # https://github.com/servo/servo/issues/11074
+        pip = _get_exec_path(PIP_NAMES, is_valid_path=check_exec_path)
+        if not pip:
+            sys.exit("Python pip is either not installed or not found in virtualenv.")
+
+        process = Popen([pip, "install", "-q", "-U", "pip"], stdout=PIPE, stderr=PIPE)
+        process.wait()
+        if process.returncode:
+            out, err = process.communicate()
+            print('Pip failed to upgrade itself properly:')
+            sys.exit('Output: %s\nError: %s' % (out, err))
+
     for req_rel_path in requirements_paths:
         req_path = os.path.join(topdir, req_rel_path)
         marker_file = req_rel_path.replace(os.path.sep, '-')
         marker_path = os.path.join(virtualenv_path, marker_file)
 
         try:
             if os.path.getmtime(req_path) + 10 < os.path.getmtime(marker_path):
                 continue