Bug 1368699 - Write .purgecaches sentinels every |mach build|. r=gps draft
authorNick Alexander <nalexander@mozilla.com>
Wed, 31 Jan 2018 09:25:12 -0800
changeset 749608 e973d065cd91e965f4103ed2732858e2e7a9c546
parent 749607 3bc8e651c03517edb797032db6ce60ed8852d9fa
child 749609 2a5b67e723f8e1654627e6e6f26704f6a4e57706
push id97458
push usernalexander@mozilla.com
push dateWed, 31 Jan 2018 20:06:51 +0000
bugs1368699, 1223748
Bug 1368699 - Write .purgecaches sentinels every |mach build|. r=gps This adds a new `post_build` step to each `BuildBackend` implementation, and uses it to write .purgecaches after every |mach build| invocation -- including after |mach build TARGET| invocations. This approach should be more robust than the existing recursive-Make based solution, which seems to not write the .purgecaches files in some situations. In addition, the recursive-Make solution does not generalize to other backends, in particular Tup. It is possible that the Tup backend will handle writing the .purgecaches sentinel as part of its regular build process, but discussions with mshal suggest that there's no convenient way for Tup to write .purgecaches only when something *changes* during the build. That is, Tup can achieve the behaviour implemented by this patch, but it's not easier to do better by not writing .purgecaches when the caches do not in fact need to be purged. I elected to bake in the special knowledge of --enable-application=browser and macOS here since this whole process is special. If we need to generalize, we could add a moz.configure option specifying the purgecaches directories, but it doesn't seem worth it right now. The ideal approach would be to determine FINAL_TARGET from the application directory, but that is determined by DIST_SUBDIR. In addition, it's not clear how to present that information to the post-build step in a build-backend agnostic manner. This will require tweaking as we migrate the macOS bundle handling to moz.build, especially in browser/app. See https://bugzilla.mozilla.org/show_bug.cgi?id=1223748, which could improve this significantly. MozReview-Commit-ID: 63KZy18D23i
--- a/python/mozbuild/mozbuild/backend/base.py
+++ b/python/mozbuild/mozbuild/backend/base.py
@@ -198,16 +198,75 @@ class BuildBackend(LoggingMixin):
         """Called when 'mach build' is executed.
         This should return the status value of a subprocess, where 0 denotes
         success and any other value is an error code. A return value of None
         indicates that the default 'make -f client.mk' should run.
         return None
+    def _write_purgecaches(self, config):
+        """Write .purgecaches sentinels.
+        The purgecaches mechanism exists to allow the platform to
+        invalidate the XUL cache (which includes some JS) at application
+        startup-time.  The application checks for .purgecaches in the
+        application directory, which varies according to
+        --enable-application.  There's a further wrinkle on macOS, where
+        the real application directory is part of a Cocoa bundle
+        produced from the regular application directory by the build
+        system.  In this case, we write to both locations, since the
+        build system recreates the Cocoa bundle from the contents of the
+        regular application directory and might remove a sentinel
+        created here.
+        """
+        app = config.substs['MOZ_BUILD_APP']
+        if app == 'mobile/android':
+            # In order to take effect, .purgecaches sentinels would need to be
+            # written to the Android device file system.
+            return
+        root = mozpath.join(config.topobjdir, 'dist', 'bin')
+        if app == 'browser':
+            root = mozpath.join(config.topobjdir, 'dist', 'bin', 'browser')
+        purgecaches_dirs = [root]
+        if app == 'browser' and 'cocoa' == config.substs['MOZ_WIDGET_TOOLKIT']:
+            bundledir = mozpath.join(config.topobjdir, 'dist',
+                                     config.substs['MOZ_MACBUNDLE_NAME'],
+                                     'Contents', 'Resources',
+                                     'browser')
+            purgecaches_dirs.append(bundledir)
+        for dir in purgecaches_dirs:
+            with open(mozpath.join(dir, '.purgecaches'), 'wt') as f:
+                f.write('\n')
+    def post_build(self, config, output, jobs, verbose, status):
+        """Called late during 'mach build' execution, after `build(...)` has finished.
+        `status` is the status value returned from `build(...)`.
+        In the case where `build` returns `None`, this is called after
+        the default `make` command has completed, with the status of
+        that command.
+        This should return the status value from `build(...)`, or the
+        status value of a subprocess, where 0 denotes success and any
+        other value is an error code.
+        If an exception is raised, |mach build| will fail with a
+        non-zero exit code.
+        """
+        self._write_purgecaches(config)
+        return status
     def _write_file(self, path=None, fh=None, mode='rU'):
         """Context manager to write a file.
         This is a glorified wrapper around FileAvoidWrite with integration to
         update the summary data on this instance.
         Example usage:
--- a/python/mozbuild/mozbuild/controller/building.py
+++ b/python/mozbuild/mozbuild/controller/building.py
@@ -1073,16 +1073,33 @@ class BuildDriver(MozbuildObject):
                 self.log(logging.WARNING, 'warning_summary',
                     {'count': len(monitor.warnings_database)},
                     '{count} compiler warnings present.')
+            # Try to run the active build backend's post-build step, if possible.
+            try:
+                config = self.config_environment
+                active_backend = config.substs.get('BUILD_BACKENDS', [None])[0]
+                if active_backend:
+                    backend_cls = get_backend_class(active_backend)(config)
+                    new_status = backend_cls.post_build(self, output, jobs, verbose, status)
+                    status = new_status
+            except Exception as ex:
+                self.log(logging.DEBUG, 'post_build', {'ex': ex},
+                         "Unable to run active build backend's post-build step; " +
+                         "failing the build due to exception: {ex}.")
+                if not status:
+                    # If the underlying build provided a failing status, pass
+                    # it through; otherwise, fail.
+                    status = 1
             monitor.finish(record_usage=status == 0)
         # Print the collected compiler warnings. This is redundant with
         # inline output from the compiler itself. However, unlike inline
         # output, this list is sorted and grouped by file, making it
         # easier to triage output.
         # Only do this if we had a successful build. If the build failed,