Bug 799648 - Part 2: Move mozbuild's log manager into mach; r=jhammel
authorGregory Szorc <gps@mozilla.com>
Wed, 10 Oct 2012 11:08:09 -0700
changeset 114434 66d59a4d5a1b76ff43dd4fe6c474e05574a60578
parent 114433 dda561124c61e11eee7fab7f286661e489264cb4
child 114435 0ccb7a8a8e5b09e98b589642fe45255996614ec5
push idunknown
push userunknown
push dateunknown
reviewersjhammel
bugs799648
milestone19.0a1
Bug 799648 - Part 2: Move mozbuild's log manager into mach; r=jhammel
python/mach/README.rst
python/mach/mach/logging.py
python/mach/mach/main.py
python/mach/mach/test/test_logger.py
python/mozbuild/README.rst
python/mozbuild/mozbuild/logger.py
python/mozbuild/mozbuild/test/test_logger.py
--- a/python/mach/README.rst
+++ b/python/mach/README.rst
@@ -78,8 +78,99 @@ Keeping Frontend Modules Small
 ------------------------------
 
 The frontend modules providing mach commands are currently all loaded when
 the mach CLI driver starts. Therefore, there is potential for *import bloat*.
 
 We want the CLI driver to load quickly. So, please delay load external modules
 until they are actually required. In other words, don't use a global
 *import* when you can import from inside a specific command's handler.
+
+Structured Logging
+==================
+
+One of the features of mach is structured logging. Instead of conventional
+logging where simple strings are logged, the internal logging mechanism logs
+all events with the following pieces of information:
+
+* A string *action*
+* A dict of log message fields
+* A formatting string
+
+Essentially, instead of assembling a human-readable string at
+logging-time, you create an object holding all the pieces of data that
+will constitute your logged event. For each unique type of logged event,
+you assign an *action* name.
+
+Depending on how logging is configured, your logged event could get
+written a couple of different ways.
+
+JSON Logging
+------------
+
+Where machines are the intended target of the logging data, a JSON
+logger is configured. The JSON logger assembles an array consisting of
+the following elements:
+
+* Decimal wall clock time in seconds since UNIX epoch
+* String *action* of message
+* Object with structured message data
+
+The JSON-serialized array is written to a configured file handle.
+Consumers of this logging stream can just perform a readline() then feed
+that into a JSON deserializer to reconstruct the original logged
+message. They can key off the *action* element to determine how to
+process individual events. There is no need to invent a parser.
+Convenient, isn't it?
+
+Logging for Humans
+------------------
+
+Where humans are the intended consumer of a log message, the structured
+log message are converted to more human-friendly form. This is done by
+utilizing the *formatting* string provided at log time. The logger
+simply calls the *format* method of the formatting string, passing the
+dict containing the message's fields.
+
+When *mach* is used in a terminal that supports it, the logging facility
+also supports terminal features such as colorization. This is done
+automatically in the logging layer - there is no need to control this at
+logging time.
+
+In addition, messages intended for humans typically prepends every line
+with the time passed since the application started.
+
+Logging HOWTO
+-------------
+
+Structured logging piggybacks on top of Python's built-in logging
+infrastructure provided by the *logging* package. We accomplish this by
+taking advantage of *logging.Logger.log()*'s *extra* argument. To this
+argument, we pass a dict with the fields *action* and *params*. These
+are the string *action* and dict of message fields, respectively. The
+formatting string is passed as the *msg* argument, like normal.
+
+If you were logging to a logger directly, you would do something like:
+
+    logger.log(logging.INFO, 'My name is {name}',
+        extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
+
+The JSON logging would produce something like:
+
+    [1339985554.306338, "my_name", {"name": "Gregory"}]
+
+Human logging would produce something like:
+
+     0.52 My name is Gregory
+
+Since there is a lot of complexity using logger.log directly, it is
+recommended to go through a wrapping layer that hides part of the
+complexity for you. The easiest way to do this is by utilizing the
+LoggingMixin:
+
+    import logging
+    from mach.mixin.logging import LoggingMixin
+
+    class MyClass(LoggingMixin):
+        def foo(self):
+             self.log(logging.INFO, 'foo_start', {'bar': True},
+                 'Foo performed. Bar: {bar}')
+
rename from python/mozbuild/mozbuild/logger.py
rename to python/mach/mach/logging.py
--- a/python/mozbuild/mozbuild/logger.py
+++ b/python/mach/mach/logging.py
@@ -1,17 +1,17 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this file,
 # You can obtain one at http://mozilla.org/MPL/2.0/.
 
-# This file contains logging functionality for mozbuild. This functionality
-# could likely be split out of mozbuild. For now, mozbuild is the only
-# consumer and thus it lives here.
+# This file contains logging functionality for mach. It essentially provides
+# support for a structured logging framework built on top of Python's built-in
+# logging framework.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 try:
     import blessings
 except ImportError:
     blessings = None
 
 import json
 import logging
--- a/python/mach/mach/main.py
+++ b/python/mach/mach/main.py
@@ -1,39 +1,39 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
 # This module provides functionality for the command-line build tool
 # (mach). It is packaged as a module because everything is a library.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import argparse
 import codecs
 import imp
 import logging
 import os
 import sys
 import traceback
 import uuid
 import sys
 
 from mozbuild.base import BuildConfig
 from mozbuild.config import ConfigSettings
-from mozbuild.logger import LoggingManager
 
-
-from mach.base import (
+from .base import (
     CommandArgument,
     CommandProvider,
     Command,
 )
 
-from mach.registrar import populate_argument_parser
+from .logging import LoggingManager
+
+from .registrar import populate_argument_parser
 
 
 # Classes inheriting from ConfigProvider that provide settings.
 # TODO this should come from auto-discovery somehow.
 SETTINGS_PROVIDERS = [
     BuildConfig,
 ]
 
rename from python/mozbuild/mozbuild/test/test_logger.py
rename to python/mach/mach/test/test_logger.py
--- a/python/mozbuild/mozbuild/test/test_logger.py
+++ b/python/mach/mach/test/test_logger.py
@@ -1,19 +1,19 @@
 # This Source Code Form is subject to the terms of the Mozilla Public
 # License, v. 2.0. If a copy of the MPL was not distributed with this
 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
 
-from __future__ import unicode_literals
+from __future__ import absolute_import, unicode_literals
 
 import logging
 import time
 import unittest
 
-from mozbuild.logger import StructuredHumanFormatter
+from mach.logger import StructuredHumanFormatter
 
 
 class DummyLogger(logging.Logger):
     def __init__(self, cb):
         logging.Logger.__init__(self, 'test')
 
         self._cb = cb
 
--- a/python/mozbuild/README.rst
+++ b/python/mozbuild/README.rst
@@ -5,98 +5,10 @@ mozbuild
 mozbuild is a Python package providing functionality used by Mozilla's
 build system.
 
 Modules Overview
 ================
 
 * mozbuild.compilation -- Functionality related to compiling. This
   includes managing compiler warnings.
-* mozbuild.logging -- Defines mozbuild's logging infrastructure.
-  mozbuild uses a structured logging backend.
 * mozbuild.testing -- Interfaces for running tests.
 
-Structured Logging
-==================
-
-One of the features of mozbuild is structured logging. Instead of
-conventional logging where simple strings are logged, the internal
-logging mechanism logs all events with the following pieces of
-information:
-
-* A string *action*
-* A dict of log message fields
-* A formatting string
-
-Essentially, instead of assembling a human-readable string at
-logging-time, you create an object holding all the pieces of data that
-will constitute your logged event. For each unique type of logged event,
-you assign an *action* name.
-
-Depending on how logging is configured, your logged event could get
-written a couple of different ways.
-
-JSON Logging
-------------
-
-Where machines are the intended target of the logging data, a JSON
-logger is configured. The JSON logger assembles an array consisting of
-the following elements:
-
-* Decimal wall clock time in seconds since UNIX epoch
-* String *action* of message
-* Object with structured message data
-
-The JSON-serialized array is written to a configured file handle.
-Consumers of this logging stream can just perform a readline() then feed
-that into a JSON deserializer to reconstruct the original logged
-message. They can key off the *action* element to determine how to
-process individual events. There is no need to invent a parser.
-Convenient, isn't it?
-
-Logging for Humans
-------------------
-
-Where humans are the intended consumer of a log message, the structured
-log message are converted to more human-friendly form. This is done by
-utilizing the *formatting* string provided at log time. The logger
-simply calls the *format* method of the formatting string, passing the
-dict containing the message's fields.
-
-When *mach* is used in a terminal that supports it, the logging facility
-also supports terminal features such as colorization. This is done
-automatically in the logging layer - there is no need to control this at
-logging time.
-
-In addition, messages intended for humans typically prepends every line
-with the time passed since the application started.
-
-Logging HOWTO
--------------
-
-Structured logging piggybacks on top of Python's built-in logging
-infrastructure provided by the *logging* package. We accomplish this by
-taking advantage of *logging.Logger.log()*'s *extra* argument. To this
-argument, we pass a dict with the fields *action* and *params*. These
-are the string *action* and dict of message fields, respectively. The
-formatting string is passed as the *msg* argument, like normal.
-
-If you were logging to a logger directly, you would do something like:
-
-    logger.log(logging.INFO, 'My name is {name}',
-        extra={'action': 'my_name', 'params': {'name': 'Gregory'}})
-
-The JSON logging would produce something like:
-
-    [1339985554.306338, "my_name", {"name": "Gregory"}]
-
-Human logging would produce something like:
-
-     0.52 My name is Gregory
-
-Since there is a lot of complexity using logger.log directly, it is
-recommended to go through a wrapping layer that hides part of the
-complexity for you. e.g.
-
-    def log(self, level, action, params, format_str):
-        self.logger.log(level, format_str,
-            extra={'action': action, 'params': params)
-