servo: Merge #11945 - Add mach build-stable to build with stable rustc (from gpoesia:master); r=SimonSapin
authorGabriel Poesia <gabriel.poesia@gmail.com>
Tue, 05 Jul 2016 07:17:48 -0700
changeset 339211 08f7a9ed15e4028cd78c612507846c00b3776cc7
parent 339210 99b7e25ae93041d8d27461d032affcf878bd2968
child 339212 04ad679bc57ca697b4cb05e5e571cf4c455c2503
push id31307
push usergszorc@mozilla.com
push dateSat, 04 Feb 2017 00:59:06 +0000
treeherdermozilla-central@94079d43835f [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersSimonSapin
servo: Merge #11945 - Add mach build-stable to build with stable rustc (from gpoesia:master); r=SimonSapin <!-- Please describe your changes on the following line: --> Added mach subcommand (build-stable) to build servo and its dependencies using a stable version of rustc. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - [X] These changes fix #11806 (github issue number if applicable). <!-- Either: --> - [X] These changes do not require tests because they're in the build process. I've manually tested both build and build-stable several times and in different orders and it seems to work. If the mach subcommands are tested automatically somewhere, I'll be happy to add a test, but I couldn't find it. This is an example of output I get, which is what I'd expect: ```plain [gpoesia@gabriel-notebook servo]$ ./mach build looking for rustc at /home/gpoesia/dev/servo/.servo/rust/2016-06-24/rustc-nightly-x86_64-unknown-linux-gnu/rustc/bin/rustc Downloading Rust compiler... Downloading Rust compiler: 100.0% Extracting Rust compiler... Rust compiler ready. Downloading Host rust library for target x86_64-unknown-linux-gnu... Downloading Host rust library for target x86_64-unknown-linux-gnu: 100.0% Extracting Rust stdlib for target x86_64-unknown-linux-gnu... Rust x86_64-unknown-linux-gnu libs ready. Build completed in 0:00:01 [gpoesia@gabriel-notebook servo]$ ./mach build Build completed in 0:00:02 [gpoesia@gabriel-notebook servo]$ ./mach build-stable looking for rustc at /home/gpoesia/dev/servo/.servo/rust/rustc-1.9.0-x86_64-unknown-linux-gnu/rustc/bin/rustc Rust compiler already downloaded. Use |bootstrap-rust --force| to download again. Rust lib for target x86_64-unknown-linux-gnu already downloaded. Use |bootstrap-rust --force| to download again. Compiling log v0.3.6 Compiling tenacious v0.2.1 Compiling traitobject v0.0.1 Compiling azure v0.4.6 (https://github.com/servo/rust-azure#4d72934a) Compiling crossbeam v0.2.9 /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/tenacious-0.2.1/src/lib.rs:1:1: 1:42 error: #[feature] may not be used on the stable release channel /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/tenacious-0.2.1/src/lib.rs:1 #![feature(plugin_registrar, box_syntax)] ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/tenacious-0.2.1/src/lib.rs:3:1: 3:27 error: #[feature] may not be used on the stable release channel /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/tenacious-0.2.1/src/lib.rs:3 #![feature(rustc_private)] ^~~~~~~~~~~~~~~~~~~~~~~~~~ error: aborting due to 2 previous errors Build failed, waiting for other jobs to finish... error: Could not compile `tenacious`. To learn more, run the command again with --verbose. Build completed in 0:00:03 [gpoesia@gabriel-notebook servo]$ ./mach build-stable looking for rustc at /home/gpoesia/dev/servo/.servo/rust/rustc-1.9.0-x86_64-unknown-linux-gnu/rustc/bin/rustc Rust compiler already downloaded. Use |bootstrap-rust --force| to download again. Rust lib for target x86_64-unknown-linux-gnu already downloaded. Use |bootstrap-rust --force| to download again. Compiling js v0.1.3 (https://github.com/servo/rust-mozjs#707bfb4f) Compiling serde_item v0.2.0 Compiling encoding-index-tradchinese v1.20141219.5 Compiling angle v0.1.0 (https://github.com/servo/angle?branch=servo#d0a2db05) /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_item-0.2.0/src/lib.rs:3:43: 3:74 error: #[feature] may not be used on the stable release channel /home/gpoesia/dev/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/serde_item-0.2.0/src/lib.rs:3 #![cfg_attr(not(feature = "with-syntex"), feature(rustc_private, plugin))] ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: aborting due to previous error Build failed, waiting for other jobs to finish... error: Could not compile `serde_item`. To learn more, run the command again with --verbose. Build completed in 0:00:37 [gpoesia@gabriel-notebook servo]$ ``` <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> Github issue: #11806 Building with current stable rust (1.9.0) still fails because of feature pragmas in some dependencies (e.g. serde_item). Source-Repo: https://github.com/servo/servo Source-Revision: 1e1db061c039d6e7635310c0c728cfb959322249
servo/ports/stable-rust/.gitignore
servo/ports/stable-rust/Cargo.toml
servo/ports/stable-rust/src/lib.rs
servo/python/servo/bootstrap_commands.py
servo/python/servo/build_commands.py
servo/python/servo/command_base.py
servo/rust-stable-version
new file mode 100644
--- /dev/null
+++ b/servo/ports/stable-rust/.gitignore
@@ -0,0 +1,1 @@
+/target
new file mode 100644
--- /dev/null
+++ b/servo/ports/stable-rust/Cargo.toml
@@ -0,0 +1,6 @@
+[package]
+name = "stable-rust"
+version = "0.0.1"
+authors = ["The Servo Project Developers"]
+
+[dependencies]
copy from servo/resources/shaders/debug_color.fs.glsl
copy to servo/ports/stable-rust/src/lib.rs
--- a/servo/resources/shaders/debug_color.fs.glsl
+++ b/servo/ports/stable-rust/src/lib.rs
@@ -1,8 +1,10 @@
 /* This Source Code Form is subject to the terms of the Mozilla Public
  * License, v. 2.0. If a copy of the MPL was not distributed with this
  * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
 
-void main(void)
-{
-    SetFragColor(vColor);
+#[cfg(test)]
+mod test {
+    #[test]
+    fn it_works() {
+    }
 }
--- a/servo/python/servo/bootstrap_commands.py
+++ b/servo/python/servo/bootstrap_commands.py
@@ -126,76 +126,100 @@ class MachCommands(CommandBase):
              category='bootstrap')
     @CommandArgument('--force', '-f',
                      action='store_true',
                      help='Force download even if a copy already exists')
     @CommandArgument('--target',
                      action='append',
                      default=[],
                      help='Download rust stdlib for specified target')
-    def bootstrap_rustc(self, force=False, target=[]):
-        rust_dir = path.join(
-            self.context.sharedir, "rust", self.rust_path())
-        date = self.rust_path().split("/")[0]
-        install_dir = path.join(self.context.sharedir, "rust", date)
+    @CommandArgument('--stable',
+                     action='store_true',
+                     help='Use stable rustc version')
+    def bootstrap_rustc(self, force=False, target=[], stable=False):
+        self.set_use_stable_rust(stable)
+        version = self.rust_version()
+        rust_path = self.rust_path()
+        if stable:
+            rust_dir = path.join(
+                self.context.sharedir, "rust", version, rust_path)
+        else:
+            rust_dir = path.join(
+                self.context.sharedir, "rust", rust_path)
+        install_dir = path.join(self.context.sharedir, "rust", version)
 
         if not force and path.exists(path.join(rust_dir, "rustc", "bin", "rustc" + BIN_SUFFIX)):
             print("Rust compiler already downloaded.", end=" ")
             print("Use |bootstrap-rust --force| to download again.")
         else:
             if path.isdir(rust_dir):
                 shutil.rmtree(rust_dir)
             os.makedirs(rust_dir)
 
-            # The Rust compiler is hosted on the nightly server under the date with a name
-            # rustc-nightly-HOST-TRIPLE.tar.gz. We just need to pull down and extract it,
+            # The nightly Rust compiler is hosted on the nightly server under the date with a name
+            # rustc-nightly-HOST-TRIPLE.tar.gz, whereas the stable compiler is named
+            # rustc-VERSION-HOST-TRIPLE.tar.gz. We just need to pull down and extract it,
             # giving a directory name that will be the same as the tarball name (rustc is
             # in that directory).
-            rustc_url = ("https://static-rust-lang-org.s3.amazonaws.com/dist/%s.tar.gz"
-                         % self.rust_path())
+            if stable:
+                rustc_url = "https://static.rust-lang.org/dist/%s.tar.gz" % rust_path
+            else:
+                rustc_url = "https://static-rust-lang-org.s3.amazonaws.com/dist/%s.tar.gz" % rust_path
             tgz_file = rust_dir + '-rustc.tar.gz'
 
             download_file("Rust compiler", rustc_url, tgz_file)
 
             print("Extracting Rust compiler...")
             extract(tgz_file, install_dir)
             print("Rust compiler ready.")
 
-        # Each Rust stdlib has a name of the form `rust-std-nightly-TRIPLE.tar.gz`, with
+        # Each Rust stdlib has a name of the form `rust-std-nightly-TRIPLE.tar.gz` for the nightly
+        # releases, or rust-std-VERSION-TRIPLE.tar.gz for stable releases, with
         # a directory of the name `rust-std-TRIPLE` inside and then a `lib` directory.
         # This `lib` directory needs to be extracted and merged with the `rustc/lib`
         # directory from the host compiler above.
-        lib_dir = path.join(install_dir, "rustc-nightly-{}".format(host_triple()),
+        nightly_suffix = "" if stable else "-nightly"
+        stable_version = "-{}".format(version) if stable else ""
+        lib_dir = path.join(install_dir,
+                            "rustc{}{}-{}".format(nightly_suffix, stable_version, host_triple()),
                             "rustc", "lib", "rustlib")
 
         # ensure that the libs for the host's target is downloaded
         host_target = host_triple()
         if host_target not in target:
             target.append(host_target)
 
         for target_triple in target:
             target_lib_dir = path.join(lib_dir, target_triple)
             if path.exists(target_lib_dir):
                 # No need to check for force. If --force the directory is already deleted
                 print("Rust lib for target {} already downloaded.".format(target_triple), end=" ")
                 print("Use |bootstrap-rust --force| to download again.")
                 continue
 
-            std_url = ("https://static-rust-lang-org.s3.amazonaws.com/dist/%s/rust-std-nightly-%s.tar.gz"
-                       % (date, target_triple))
-            tgz_file = install_dir + ('rust-std-nightly-%s.tar.gz' % target_triple)
+            if self.use_stable_rust():
+                std_url = ("https://static.rust-lang.org/dist/rust-std-%s-%s.tar.gz"
+                           % (version, target_triple))
+                tgz_file = install_dir + ('rust-std-%s-%s.tar.gz' % (version, target_triple))
+            else:
+                std_url = ("https://static-rust-lang-org.s3.amazonaws.com/dist/%s/rust-std-nightly-%s.tar.gz"
+                           % (version, target_triple))
+                tgz_file = install_dir + ('rust-std-nightly-%s.tar.gz' % target_triple)
 
             download_file("Host rust library for target %s" % target_triple, std_url, tgz_file)
             print("Extracting Rust stdlib for target %s..." % target_triple)
             extract(tgz_file, install_dir)
-            shutil.copytree(path.join(install_dir, "rust-std-nightly-%s" % target_triple,
+            shutil.copytree(path.join(install_dir,
+                                      "rust-std%s%s-%s" % (nightly_suffix, stable_version, target_triple),
                                       "rust-std-%s" % target_triple, "lib", "rustlib", target_triple),
-                            path.join(install_dir, "rustc-nightly-%s" % host_triple(),
+                            path.join(install_dir,
+                                      "rustc%s%s-%s" % (nightly_suffix, stable_version, host_triple),
                                       "rustc", "lib", "rustlib", target_triple))
-            shutil.rmtree(path.join(install_dir, "rust-std-nightly-%s" % target_triple))
+            shutil.rmtree(path.join(install_dir,
+                          "rust-std%s%s-%s" % (nightly_suffix, stable_version, target_triple)))
 
             print("Rust {} libs ready.".format(target_triple))
 
     @Command('bootstrap-rust-docs',
              description='Download the Rust documentation',
              category='bootstrap')
     @CommandArgument('--force', '-f',
                      action='store_true',
--- a/servo/python/servo/build_commands.py
+++ b/servo/python/servo/build_commands.py
@@ -260,16 +260,53 @@ class MachCommands(CommandBase):
                 pass
 
         # Generate Desktop Notification if elapsed-time > some threshold value
         notify_build_done(elapsed)
 
         print("Build completed in %s" % format_duration(elapsed))
         return status
 
+    @Command('build-stable',
+             description='Build Servo using stable rustc',
+             category='build')
+    @CommandArgument('--target', '-t',
+                     default=None,
+                     help='Cross compile for given target platform')
+    @CommandArgument('--release', '-r',
+                     action='store_true',
+                     help='Build in release mode')
+    @CommandArgument('--dev', '-d',
+                     action='store_true',
+                     help='Build in development mode')
+    @CommandArgument('--jobs', '-j',
+                     default=None,
+                     help='Number of jobs to run in parallel')
+    @CommandArgument('--features',
+                     default=None,
+                     help='Space-separated list of features to also build',
+                     nargs='+')
+    @CommandArgument('--android',
+                     default=None,
+                     action='store_true',
+                     help='Build for Android')
+    @CommandArgument('--debug-mozjs',
+                     default=None,
+                     action='store_true',
+                     help='Enable debug assertions in mozjs')
+    @CommandArgument('--verbose', '-v',
+                     action='store_true',
+                     help='Print verbose output')
+    @CommandArgument('params', nargs='...',
+                     help="Command-line arguments to be passed through to Cargo")
+    def build_stable(self, target=None, release=False, dev=False, jobs=None,
+                     features=None, android=None, verbose=False, debug_mozjs=False, params=None):
+        self.set_use_stable_rust()
+        self.build(target, release, dev, jobs, features, android, verbose, debug_mozjs, params)
+
     @Command('build-cef',
              description='Build the Chromium Embedding Framework library',
              category='build')
     @CommandArgument('--jobs', '-j',
                      default=None,
                      help='Number of jobs to run in parallel')
     @CommandArgument('--verbose', '-v',
                      action='store_true',
--- a/servo/python/servo/command_base.py
+++ b/servo/python/servo/command_base.py
@@ -179,19 +179,17 @@ class CommandBase(object):
         resolverelative("tools", "cargo-home-dir")
 
         context.sharedir = self.config["tools"]["cache-dir"]
 
         self.config["tools"].setdefault("system-rust", False)
         self.config["tools"].setdefault("system-cargo", False)
         self.config["tools"].setdefault("rust-root", "")
         self.config["tools"].setdefault("cargo-root", "")
-        if not self.config["tools"]["system-rust"]:
-            self.config["tools"]["rust-root"] = path.join(
-                context.sharedir, "rust", self.rust_path())
+        self.set_use_stable_rust(False)
         if not self.config["tools"]["system-cargo"]:
             self.config["tools"]["cargo-root"] = path.join(
                 context.sharedir, "cargo", self.cargo_build_id())
         self.config["tools"].setdefault("rustc-with-gold", get_env_bool("SERVO_RUSTC_WITH_GOLD", True))
 
         self.config.setdefault("build", {})
         self.config["build"].setdefault("android", False)
         self.config["build"].setdefault("mode", "")
@@ -200,26 +198,43 @@ class CommandBase(object):
 
         self.config.setdefault("android", {})
         self.config["android"].setdefault("sdk", "")
         self.config["android"].setdefault("ndk", "")
         self.config["android"].setdefault("toolchain", "")
         self.config["android"].setdefault("platform", "android-18")
         self.config["android"].setdefault("target", "arm-linux-androideabi")
 
-    _rust_path = None
+    _use_stable_rust = False
+    _rust_version = None
+    _rust_version_is_stable = False
     _cargo_build_id = None
 
+    def set_use_stable_rust(self, use_stable_rust=True):
+        self._use_stable_rust = use_stable_rust
+        if not self.config["tools"]["system-rust"]:
+            self.config["tools"]["rust-root"] = path.join(
+                self.context.sharedir, "rust", self.rust_path())
+
+    def use_stable_rust(self):
+        return self._use_stable_rust
+
     def rust_path(self):
-        if self._rust_path is None:
-            filename = path.join(self.context.topdir, "rust-nightly-date")
+        if self._use_stable_rust:
+            return "rustc-%s-%s" % (self.rust_version(), host_triple())
+        else:
+            return "%s/rustc-nightly-%s" % (self.rust_version(), host_triple())
+
+    def rust_version(self):
+        if self._rust_version is None or self._use_stable_rust != self._rust_version_is_stable:
+            filename = path.join(self.context.topdir,
+                                 "rust-stable-version" if self._use_stable_rust else "rust-nightly-date")
             with open(filename) as f:
-                date = f.read().strip()
-            self._rust_path = ("%s/rustc-nightly-%s" % (date, host_triple()))
-        return self._rust_path
+                self._rust_version = f.read().strip()
+        return self._rust_version
 
     def cargo_build_id(self):
         if self._cargo_build_id is None:
             filename = path.join(self.context.topdir, "cargo-nightly-build")
             with open(filename) as f:
                 self._cargo_build_id = f.read().strip()
         return self._cargo_build_id
 
@@ -312,17 +327,19 @@ class CommandBase(object):
             extra_path += [
                 path.join(self.config["tools"]["cargo-root"], "bin")]
 
         if extra_path:
             env["PATH"] = "%s%s%s" % (os.pathsep.join(extra_path), os.pathsep, env["PATH"])
 
         env["CARGO_HOME"] = self.config["tools"]["cargo-home-dir"]
 
-        if "CARGO_TARGET_DIR" not in env:
+        if self.use_stable_rust():
+            env["CARGO_TARGET_DIR"] = path.join(self.context.topdir, "ports/stable-rust/target")
+        elif "CARGO_TARGET_DIR" not in env:
             env["CARGO_TARGET_DIR"] = path.join(self.context.topdir, "target")
 
         if extra_lib:
             if sys.platform == "darwin":
                 env["DYLD_LIBRARY_PATH"] = "%s%s%s" % \
                                            (os.pathsep.join(extra_lib),
                                             os.pathsep,
                                             env.get("DYLD_LIBRARY_PATH", ""))
@@ -421,17 +438,18 @@ class CommandBase(object):
         base_target_path = path.join(rust_root, "rustc", "lib", "rustlib")
         target_exists = True
         if target is not None:
             target_path = path.join(base_target_path, target)
             target_exists = path.exists(target_path)
 
         if not (self.config['tools']['system-rust'] or (rustc_binary_exists and target_exists)):
             print("looking for rustc at %s" % (rustc_path))
-            Registrar.dispatch("bootstrap-rust", context=self.context, target=filter(None, [target]))
+            Registrar.dispatch("bootstrap-rust", context=self.context, target=filter(None, [target]),
+                               stable=self._use_stable_rust)
 
         cargo_path = path.join(self.config["tools"]["cargo-root"], "cargo", "bin",
                                "cargo" + BIN_SUFFIX)
         cargo_binary_exists = path.exists(cargo_path)
 
         if not self.config["tools"]["system-cargo"] and not cargo_binary_exists:
             Registrar.dispatch("bootstrap-cargo", context=self.context)
 
new file mode 100644
--- /dev/null
+++ b/servo/rust-stable-version
@@ -0,0 +1,1 @@
+1.9.0