Bug 1483941: add documentation about build telemetry r=ted
authorConnor Sheehan <sheehan@mozilla.com>
Wed, 10 Oct 2018 19:58:47 +0000
changeset 499224 122ac3c6e1dc29aeb824845419658dfd6c45604a
parent 499223 7a275e7abf9b88e0d70a14e5a3dea3a7a8ce21e0
child 499225 b9222b391f8960132c31d050d1843ffb32ac1693
push id1864
push userffxbld-merge
push dateMon, 03 Dec 2018 15:51:40 +0000
treeherdermozilla-release@f040763d99ad [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersted
bugs1483941
milestone64.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1483941: add documentation about build telemetry r=ted This commit adds some documentation about build telemetry to the build system documentation. This doc should be used as a reference during opt-in to build system telemetry, and includes paths to the relevant files for users to audit independently. Documentation regarding the schema types and object properties was generated by the `jsonschema2rst` tool, available on PyPI. Differential Revision: https://phabricator.services.mozilla.com/D8030
build/docs/index.rst
build/docs/telemetry.rst
--- a/build/docs/index.rst
+++ b/build/docs/index.rst
@@ -25,16 +25,17 @@ Important Concepts
    preprocessor
    jar-manifests
    defining-binaries
    toolchains
    locales
    rust
    sparse
    Support for projects building with GN <gn>
+   telemetry
 
 integrated development environment (IDE)
 ========================================
 .. toctree::
    :maxdepth: 1
 
    androideclipse
    cppeclipse
new file mode 100644
--- /dev/null
+++ b/build/docs/telemetry.rst
@@ -0,0 +1,323 @@
+.. _buildtelemetry:
+
+===============
+Build Telemetry
+===============
+
+The build system (specifically, all the build tooling hooked
+up to ``./mach``) has been configured to collect metrics data
+points for various build system actions. This data helps drive
+team planning for the build team and ensure that resources are
+applied to build processes that need them most. You can opt-in
+to send telemetry to Mozilla during `./mach bootstrap`.
+
+The build telemetry schema can be found in-tree under
+``python/mozbuild/mozbuild/telemetry.py`` in Voluptuous schema
+format. You can use the ``export_telemetry_schema.py`` script in
+that same directory to get the schema in JSON-schema format.
+Details of the schema are specified below:
+
+
+
+.. _telemetry.json#/:
+
+telemetry
+=========
+
+:type: ``object``
+
+:Required: :ref:`telemetry.json#/properties/argv`, :ref:`telemetry.json#/properties/build_opts`, :ref:`telemetry.json#/properties/client_id`, :ref:`telemetry.json#/properties/command`, :ref:`telemetry.json#/properties/duration_ms`, :ref:`telemetry.json#/properties/success`, :ref:`telemetry.json#/properties/system`, :ref:`telemetry.json#/properties/time`
+
+**Properties:** :ref:`telemetry.json#/properties/argv`, :ref:`telemetry.json#/properties/build_opts`, :ref:`telemetry.json#/properties/client_id`, :ref:`telemetry.json#/properties/command`, :ref:`telemetry.json#/properties/duration_ms`, :ref:`telemetry.json#/properties/exception`, :ref:`telemetry.json#/properties/file_types_changed`, :ref:`telemetry.json#/properties/success`, :ref:`telemetry.json#/properties/system`, :ref:`telemetry.json#/properties/time`
+
+
+.. _telemetry.json#/properties/argv:
+
+argv
+++++
+
+Full mach commandline. If the commandline contains absolute paths they will be sanitized.
+
+:type: ``array``
+
+.. container:: sub-title
+
+ Every element of **argv**  is:
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/build_opts:
+
+build_opts
+++++++++++
+
+Selected build options
+
+:type: ``object``
+
+**Properties:** :ref:`telemetry.json#/properties/build_opts/properties/artifact`, :ref:`telemetry.json#/properties/build_opts/properties/ccache`, :ref:`telemetry.json#/properties/build_opts/properties/compiler`, :ref:`telemetry.json#/properties/build_opts/properties/debug`, :ref:`telemetry.json#/properties/build_opts/properties/icecream`, :ref:`telemetry.json#/properties/build_opts/properties/opt`, :ref:`telemetry.json#/properties/build_opts/properties/sccache`
+
+
+.. _telemetry.json#/properties/build_opts/properties/artifact:
+
+artifact
+########
+
+true if --enable-artifact-builds
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/build_opts/properties/ccache:
+
+ccache
+######
+
+true if ccache is in use (--with-ccache)
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/build_opts/properties/compiler:
+
+compiler
+########
+
+The compiler type in use (CC_TYPE)
+
+**Allowed values:**
+
+- clang
+- clang-cl
+- gcc
+- msvc
+
+
+.. _telemetry.json#/properties/build_opts/properties/debug:
+
+debug
+#####
+
+true if build is debug (--enable-debug)
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/build_opts/properties/icecream:
+
+icecream
+########
+
+true if icecream in use
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/build_opts/properties/opt:
+
+opt
+###
+
+true if build is optimized (--enable-optimize)
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/build_opts/properties/sccache:
+
+sccache
+#######
+
+true if ccache in use is sccache
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/client_id:
+
+client_id
++++++++++
+
+A UUID to uniquely identify a client
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/command:
+
+command
++++++++
+
+The mach command that was invoked
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/duration_ms:
+
+duration_ms
++++++++++++
+
+Command duration in milliseconds
+
+:type: ``number``
+
+
+.. _telemetry.json#/properties/exception:
+
+exception
++++++++++
+
+If a Python exception was encountered during the execution of the command, this value contains the result of calling `repr` on the exception object.
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/file_types_changed:
+
+file_types_changed
+++++++++++++++++++
+
+This array contains a list of objects with {ext, count} properties giving the count of files changed since the last invocation grouped by file type
+
+:type: ``array``
+
+.. container:: sub-title
+
+ Every element of **file_types_changed**  is:
+
+:type: ``object``
+
+:Required: :ref:`telemetry.json#/properties/file_types_changed/items/properties/count`, :ref:`telemetry.json#/properties/file_types_changed/items/properties/ext`
+
+**Properties:** :ref:`telemetry.json#/properties/file_types_changed/items/properties/count`, :ref:`telemetry.json#/properties/file_types_changed/items/properties/ext`
+
+
+.. _telemetry.json#/properties/file_types_changed/items/properties/count:
+
+count
+#####
+
+Count of changed files with this extension
+
+:type: ``number``
+
+
+.. _telemetry.json#/properties/file_types_changed/items/properties/ext:
+
+ext
+###
+
+File extension
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/success:
+
+success
++++++++
+
+true if the command succeeded
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/system:
+
+system
+++++++
+
+:type: ``object``
+
+:Required: :ref:`telemetry.json#/properties/system/properties/os`
+
+**Properties:** :ref:`telemetry.json#/properties/system/properties/cpu_brand`, :ref:`telemetry.json#/properties/system/properties/drive_is_ssd`, :ref:`telemetry.json#/properties/system/properties/logical_cores`, :ref:`telemetry.json#/properties/system/properties/memory_gb`, :ref:`telemetry.json#/properties/system/properties/os`, :ref:`telemetry.json#/properties/system/properties/physical_cores`, :ref:`telemetry.json#/properties/system/properties/virtual_machine`
+
+
+.. _telemetry.json#/properties/system/properties/cpu_brand:
+
+cpu_brand
+#########
+
+CPU brand string from CPUID
+
+:type: ``string``
+
+
+.. _telemetry.json#/properties/system/properties/drive_is_ssd:
+
+drive_is_ssd
+############
+
+true if the source directory is on a solid-state disk
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/system/properties/logical_cores:
+
+logical_cores
+#############
+
+Number of logical CPU cores present
+
+:type: ``number``
+
+
+.. _telemetry.json#/properties/system/properties/memory_gb:
+
+memory_gb
+#########
+
+System memory in GB
+
+:type: ``number``
+
+
+.. _telemetry.json#/properties/system/properties/os:
+
+os
+##
+
+Operating system
+
+**Allowed values:**
+
+- windows
+- macos
+- linux
+- other
+
+
+.. _telemetry.json#/properties/system/properties/physical_cores:
+
+physical_cores
+##############
+
+Number of physical CPU cores present
+
+:type: ``number``
+
+
+.. _telemetry.json#/properties/system/properties/virtual_machine:
+
+virtual_machine
+###############
+
+true if the OS appears to be running in a virtual machine
+
+:type: ``boolean``
+
+
+.. _telemetry.json#/properties/time:
+
+time
+++++
+
+Time at which this event happened
+
+:type: ``string``
+
+:format: ``date-time``