Bug 799648 - Part 2: Move mozbuild's log manager into mach; r=jhammel
authorGregory Szorc <gps@mozilla.com>
Wed, 10 Oct 2012 11:08:09 -0700
changeset 109794 66d59a4d5a1b76ff43dd4fe6c474e05574a60578
parent 109793 dda561124c61e11eee7fab7f286661e489264cb4
child 109795 0ccb7a8a8e5b09e98b589642fe45255996614ec5
push id23654
push usergszorc@mozilla.com
push dateWed, 10 Oct 2012 18:09:39 +0000
treeherdermozilla-central@c0977caba1d6 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersjhammel
bugs799648
milestone19.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 799648 - Part 2: Move mozbuild's log manager into mach; r=jhammel
python/mach/README.rst
python/mach/mach/logging.py
python/mach/mach/main.py
python/mach/mach/test/test_logger.py
python/mozbuild/README.rst
python/mozbuild/mozbuild/logger.py
python/mozbuild/mozbuild/test/test_logger.py
--- a/python/mach/README.rst
+++ b/python/mach/README.rst
@@ -78,8 +78,99 @@ Keeping Frontend Modules Small
 ------------------------------
 
 The frontend modules providing mach commands are currently all loaded when
 the mach CLI driver starts. Therefore, there is potential for *import bloat*.
 
 We want the CLI driver to load quickly. So, please delay load external modules
 until they are actually required. In other words, don't use a global
 *import* when you can import from inside a specific command's handler.
+
+Structured Logging
+==================
+
+One of the features of mach is structured logging. Instead of conventional
+logging where simple strings are logged, the internal logging mechanism logs
+all events with the following pieces of information:
+
+* A string *action*
+* A dict of log message fields
+* A formatting string
+
+Essentially, instead of assembling a human-readable string at
+logging-time, you create an object holding all the pieces of data that
+will constitute your logged event. For each unique type of logged event,
+you assign an *action* name.
+
+Depending on how logging is configured, your logged event could get
+written a couple of different ways.
+
+JSON Logging
+------------
+
+Where machines are the intended target of the logging data, a JSON
+logger is configured. The JSON logger assembles an array consisting of
+the following elements:
+
+* Decimal wall clock time in seconds since UNIX epoch
+* String *action* of message
+* Object with structured message data
+
+The JSON-serialized array is written to a configured file handle.
+Consumers of this logging stream can just perform a readline() then feed
+that into a JSON deserializer to reconstruct the original logged
+message. They can key off the *action* element to determine how to
+process individual events. There is no need to invent a parser.
+Convenient, isn't it?
+
+Logging for Humans
+------------------
+
+Where humans are the intended consumer of a log message, the structured
+log message are converted to more human-friendly form. This is done by
+utilizing the *formatting* string provided at log time. The logger
+simply calls the *format* method of the formatting string, passing the
+dict containing the message's fields.
+
+When *mach* is used in a terminal that supports it, the logging facility
+also supports terminal features such as colorization. This is done
+automatically in the logging layer - there is no need to control this at
+logging time.
+
+In addition, messages intended for humans typically prepends every line
+with the time passed since the application started.
+
+Logging HOWTO
+-------------
+
+Structured logging piggybacks on top of Python's built-in logging
+infrastructure provided by the *logging* package. We accomplish this by
+taking advantage of *logging.Logger.log()*'s *extra* argument. To this
+argument, we pass a dict with the fields *action* and *params*. These
+are the string *action* and dict of message fields, respectively. The
+formatting string is passed as the *msg* argument, like normal.
+
+If you were logging to a logger directly, you would do something like:
+
+    logger.log(logging.INFO, 'My name is {name}',
+        extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
+
+The JSON logging would produce something like:
+
+    [1339985554.306338, "my_name", {"name": "Gregory"}]
+
+Human logging would produce something like:
+
+     0.52 My name is Gregory
+
+Since there is a lot of complexity using logger.log directly, it is
+recommended to go through a wrapping layer that hides part of the
+complexity for you. The easiest way to do this is by utilizing the
+LoggingMixin:
+
+    import logging
+    from mach.mixin.logging import LoggingMixin
+
+    class MyClass(LoggingMixin):
+        def foo(self):
+             self.log(logging.INFO, 'foo_start', {'bar': True},
+                 'Foo performed. Bar: {bar}')
+
rename from python/mozbuild/mozbuild/logger.py
rename to python/mach/mach/logging.py
--- a/python/mozbuild/mozbuild/logger.py
+++ b/python/mach/mach/logging.py
@@ -1,17 +1,17 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this file,
 # You can obtain one at http://mozilla.org/MPL/2.0/.
 
-# This file contains logging functionality for mozbuild. This functionality
-# could likely be split out of mozbuild. For now, mozbuild is the only
-# consumer and thus it lives here.
+# This file contains logging functionality for mach. It essentially provides
+# support for a structured logging framework built on top of Python's built-in
+# logging framework.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 try:
     import blessings
 except ImportError:
     blessings = None
 
 import json
 import logging
--- a/python/mach/mach/main.py
+++ b/python/mach/mach/main.py
@@ -1,39 +1,39 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
 # This module provides functionality for the command-line build tool
 # (mach). It is packaged as a module because everything is a library.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import argparse
 import codecs
 import imp
 import logging
 import os
 import sys
 import traceback
 import uuid
 import sys
 
 from mozbuild.base import BuildConfig
 from mozbuild.config import ConfigSettings
-from mozbuild.logger import LoggingManager
 
-
-from mach.base import (
+from .base import (
     CommandArgument,
     CommandProvider,
     Command,
 )
 
-from mach.registrar import populate_argument_parser
+from .logging import LoggingManager
+
+from .registrar import populate_argument_parser
 
 
 # Classes inheriting from ConfigProvider that provide settings.
 # TODO this should come from auto-discovery somehow.
 SETTINGS_PROVIDERS = [
     BuildConfig,
 ]
 
rename from python/mozbuild/mozbuild/test/test_logger.py
rename to python/mach/mach/test/test_logger.py
--- a/python/mozbuild/mozbuild/test/test_logger.py
+++ b/python/mach/mach/test/test_logger.py
@@ -1,19 +1,19 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import logging
 import time
 import unittest
 
-from mozbuild.logger import StructuredHumanFormatter
+from mach.logger import StructuredHumanFormatter
 
 
 class DummyLogger(logging.Logger):
     def __init__(self, cb):
         logging.Logger.__init__(self, 'test')
 
         self._cb = cb
 
--- a/python/mozbuild/README.rst
+++ b/python/mozbuild/README.rst
@@ -5,98 +5,10 @@ mozbuild
 mozbuild is a Python package providing functionality used by Mozilla's
 build system.
 
 Modules Overview
 ================
 
 * mozbuild.compilation -- Functionality related to compiling. This
   includes managing compiler warnings.
-* mozbuild.logging -- Defines mozbuild's logging infrastructure.
-  mozbuild uses a structured logging backend.
 * mozbuild.testing -- Interfaces for running tests.
 
-Structured Logging
-==================
-
-One of the features of mozbuild is structured logging. Instead of
-conventional logging where simple strings are logged, the internal
-logging mechanism logs all events with the following pieces of
-information:
-
-* A string *action*
-* A dict of log message fields
-* A formatting string
-
-Essentially, instead of assembling a human-readable string at
-logging-time, you create an object holding all the pieces of data that
-will constitute your logged event. For each unique type of logged event,
-you assign an *action* name.
-
-Depending on how logging is configured, your logged event could get
-written a couple of different ways.
-
-JSON Logging
-------------
-
-Where machines are the intended target of the logging data, a JSON
-logger is configured. The JSON logger assembles an array consisting of
-the following elements:
-
-* Decimal wall clock time in seconds since UNIX epoch
-* String *action* of message
-* Object with structured message data
-
-The JSON-serialized array is written to a configured file handle.
-Consumers of this logging stream can just perform a readline() then feed
-that into a JSON deserializer to reconstruct the original logged
-message. They can key off the *action* element to determine how to
-process individual events. There is no need to invent a parser.
-Convenient, isn't it?
-
-Logging for Humans
-------------------
-
-Where humans are the intended consumer of a log message, the structured
-log message are converted to more human-friendly form. This is done by
-utilizing the *formatting* string provided at log time. The logger
-simply calls the *format* method of the formatting string, passing the
-dict containing the message's fields.
-
-When *mach* is used in a terminal that supports it, the logging facility
-also supports terminal features such as colorization. This is done
-automatically in the logging layer - there is no need to control this at
-logging time.
-
-In addition, messages intended for humans typically prepends every line
-with the time passed since the application started.
-
-Logging HOWTO
--------------
-
-Structured logging piggybacks on top of Python's built-in logging
-infrastructure provided by the *logging* package. We accomplish this by
-taking advantage of *logging.Logger.log()*'s *extra* argument. To this
-argument, we pass a dict with the fields *action* and *params*. These
-are the string *action* and dict of message fields, respectively. The
-formatting string is passed as the *msg* argument, like normal.
-
-If you were logging to a logger directly, you would do something like:
-
-    logger.log(logging.INFO, 'My name is {name}',
-        extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
-
-The JSON logging would produce something like:
-
-    [1339985554.306338, "my_name", {"name": "Gregory"}]
-
-Human logging would produce something like:
-
-     0.52 My name is Gregory
-
-Since there is a lot of complexity using logger.log directly, it is
-recommended to go through a wrapping layer that hides part of the
-complexity for you. e.g.
-
-    def log(self, level, action, params, format_str):
-        self.logger.log(level, format_str,
-            extra={'action': action, 'params': params)
-