Bug 1107193 - Add a script that makes it easy to run the static rooting analysis. r=sfink.
authorJason Orendorff <jorendorff@mozilla.com>
Wed, 03 Dec 2014 20:02:02 -0600
changeset 225791 d7b40d5a4577f965c6fe1bc7f99dc9321dccdf57
parent 225790 f8d8f84fc7bfd64727068e28783cc97bf6736bd4
child 225792 af81a26dae604b133805465193d647cc86a49910
push id28175
push userryanvm@gmail.com
push dateMon, 26 Jan 2015 21:33:41 +0000
treeherdermozilla-central@a6f037b538ed [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewerssfink
bugs1107193
milestone38.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1107193 - Add a script that makes it easy to run the static rooting analysis. r=sfink.
js/src/devtools/rootAnalysis/README.txt
js/src/devtools/rootAnalysis/run-analysis.sh
--- a/js/src/devtools/rootAnalysis/README.txt
+++ b/js/src/devtools/rootAnalysis/README.txt
@@ -1,39 +1,49 @@
+# Spidermonkey JSAPI rooting analysis
+
 This directory contains scripts and a makefile for running Brian Hackett's
 static GC rooting analysis on a JS source directory.
 
-To use it:
+To use it on SpiderMonkey:
+
+1.  Be on Fedora/CentOS/RedHat Linux x86_64.
 
-1. Download and compile sixgill. Make sure the gcc plugin is enabled. (The
-   configure output will tell you.)
-
-  - [sfink] I needed a couple of patches to get it work on Fedora 16/17/18.
-    Ask me if you need them.
+    (Specifically, the prebuilt GCC **won't work on Ubuntu**
+    without the `CFLAGS` and `CXXFLAGS` settings from
+    http://trac.wildfiregames.com/wiki/StaticRootingAnalysis .)
 
-2. Compile an optimized JS shell that includes the patch at
-   <http://people.mozilla.org/~sfink/data/bug-835552-cwd-snarf>. This does not
-   need to be in the same source tree as you are running these scripts from.
-   Remember the full path to the resulting JS binary; we'll call it $JS_SHELL
-   below.
+2.  Have the Gecko build prerequisites installed.
+
+3.  In this directory, run these commands.
 
-3. |make clean| in the objdir of the JS source tree that you're going to be
-   analyzing. (These analysis scripts will default to the tree they are within,
-   but you can point them at another tree.)
+        mkdir builddir
+        cd builddir
+        ../run-analysis.sh
 
-4. in $objdir/js/src/devtools/analysis, |make JS=$JS_SHELL
-   SIXGILL=.../path/to/sixgill...|. You may need one or more of the following
-   additional settings in addition to the |JS| already given:
+`run-analysis.sh` is kind of like `configure` and `make` combined:
+the build directory can be wherever you want
+and you can name it whatever you want.
+(You could just run it right here in the source tree, and it would work,
+but don't do that -- it spits out files all over the place and
+then you'd have to clean up your source tree later.)
 
-   - JSOBJDIR: if you are analyzing a different source tree, set this to the
-     objdir of the tree you want to analyze.
+Output goes to `hazards.txt` in the builddir.
+
+To use this analysis on any other codebase,
+make a copy of `run-analysis.sh` and adapt it for your code.
+
 
-   - ANALYSIS_SCRIPT_DIR: by default, the *.js files within the directory
-     containing this README will be used, but you can point to a different
-     directory full. At the time of this writing, there are some changes not in
-     bhackett's git repo that are necessary, and you'll also need the
-     gen-hazards.sh shell script.
+## Overview of what is going on here
+
+So what does `run-analysis.sh` actually do?
+
+1.  **It insecurely downloads software over HTTP.** Yeah.
+    See `run-analysis.sh` for details.
 
-   - JOBS: set this to the number of parallel jobs you'd like to run the final
-     analysis pass with. This defaults to 6, somewhat randomly, which gave me a
-     large speedup even on a machine with only 2 cores.
+2.  It runs `run_complete`, a Perl script, which builds the target
+    codebase with a custom hacked GCC, generating a few database files
+    containing (among other data) the full call graph.
 
-The results will be in rootingHazards.txt
+3.  Then it runs `analyze.py`, a Python script, which runs all the scripts
+    which actually perform the analysis -- the tricky parts.
+    (Those scripts are written in JS.)
+
new file mode 100755
--- /dev/null
+++ b/js/src/devtools/rootAnalysis/run-analysis.sh
@@ -0,0 +1,85 @@
+# setup.sh - Run the rooting analysis on SpiderMonkey. See `README.txt` for usage.
+#
+# This script is based on the wiki page:
+# http://trac.wildfiregames.com/wiki/StaticRootingAnalysis
+
+set -eu
+
+BUILD_DIR="$PWD"
+ANALYSIS_SCRIPTDIR="$(dirname $0)"
+MOZILLA_SRCDIR="$(cd $ANALYSIS_SCRIPTDIR && (hg root || git rev-parse --show-toplevel))"
+
+
+# Requirements
+# ============
+#
+# Download and unpack the Sixgill plugin binaries.
+# (`wget -c` skips the download if you've already got the file.)
+#
+# This insecurely downloads software over HTTP. Sorry.
+#
+# The alternative is building your own Sixgill. That can be a pain and you may
+# need some patches to get it to work on your Linux distribution. Ask sfink for
+# details.
+
+mkdir -p downloads
+(cd downloads && wget -c http://people.mozilla.org/~sfink/data/hazards-sixgill.tar.xz)
+tar xf downloads/hazards-sixgill.tar.xz
+
+# Download and unpack GCC binaries compatible with the Sixgill plugin.
+(cd downloads && wget -c http://people.mozilla.org/~sfink/data/hazards-gcc4.7.tar.xz)
+tar xf downloads/hazards-gcc4.7.tar.xz
+
+
+# Generate raw data (.xdb files)
+# ==============================
+#
+# The first step is to generate the .xdb files that contain the information
+# needed by the analysis. This is done by compiling SpiderMonkey with the
+# sixgill plugin enabled. The plugin creates .xdb files which the analysis
+# consumes.
+
+PATH=$BUILD_DIR/sixgill/usr/bin:$PATH
+export PATH
+GCCDIR=$BUILD_DIR/gcc/bin
+export GCCDIR
+
+# Create a SpiderMonkey build directory and run configure.
+mkdir -p spidermonkey-analysis
+(cd spidermonkey-analysis && \
+    $MOZILLA_SRCDIR/js/src/configure --enable-optimize)
+
+# Make SpiderMonkey.
+$MOZILLA_SRCDIR/js/src/devtools/rootAnalysis/run_complete \
+    --build-root=$BUILD_DIR/spidermonkey-analysis \
+    --binaries=$BUILD_DIR/sixgill/usr/bin \
+    --wrap-dir=$BUILD_DIR/sixgill/usr/libexec/sixgill/scripts/wrap_gcc \
+    --buildcommand='make' \
+    --foreground \
+    --no-logs \
+    .
+
+
+# Run the analysis
+# ================
+
+# Build *another* copy of SpiderMonkey, using the system C++ compiler, without
+# Sixgill. This is what we use to run the analysis. (We don't let you skip this
+# step by setting a $JS environment variable or something, because you need
+# ctypes. Relax and spin a build.  Get yourself a cup of tea.)
+mkdir -p spidermonkey-opt
+(cd spidermonkey-opt && \
+    $MOZILLA_SRCDIR/js/src/configure --enable-optimize --enable-ctypes --enable-nspr-build && \
+    make -j8)
+JS="$BUILD_DIR/spidermonkey-opt/dist/bin/js"
+
+# Write a config file used by analyze.py.
+rm -f defaults.py
+echo "objdir = '${BUILD_DIR}/spidermonkey-analysis'" >> defaults.py
+echo "sixgill = '${BUILD_DIR}/sixgill/usr/libexec/sixgill'" >> defaults.py
+echo "sixgill_bin = '${BUILD_DIR}/sixgill/usr/bin'" >> defaults.py
+echo "js = '${JS}'" >> defaults.py
+echo "analysis_scriptdir = '${ANALYSIS_SCRIPTDIR}'" >> defaults.py
+
+# Run the script that runs the scripts that do the analysis.
+python2.7 "${MOZILLA_SRCDIR}/js/src/devtools/rootAnalysis/analyze.py" -j 8 callgraph