Bug 1286075: add how-tos covering new functionality; r=gps
authorDustin J. Mitchell <dustin@mozilla.com>
Fri, 09 Sep 2016 21:29:27 +0000
changeset 313721 550404a4b851662a64fd9355fe176d932fcfe0eb
parent 313720 e96a7223e0d0032ac9d17c477393d5f596349317
child 313722 9d0e4afed75bc8a2227515cde317e369ea999d6d
push id20528
push userryanvm@gmail.com
push dateWed, 14 Sep 2016 00:48:16 +0000
treeherderfx-team@8a996d7c71a2 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
Bug 1286075: add how-tos covering new functionality; r=gps MozReview-Commit-ID: 5l6v81UKskC
--- a/taskcluster/docs/how-tos.rst
+++ b/taskcluster/docs/how-tos.rst
@@ -39,17 +39,17 @@ 1. Find a recent decision task on the pr
 2. Run one of the ``mach taskgraph`` subcommands (see :doc:`taskgraph`) to
    generate a baseline against which to measure your changes.  For example:
    .. code-block:: none
        ./mach taskgraph --json -p parameters.yml tasks > old-tasks.json
-3. Make your modifications under ``tsakcluster/``.
+3. Make your modifications under ``taskcluster/``.
 4. Run the same ``mach taskgraph`` command, sending the output to a new file,
    and use ``diff`` to compare the old and new files.  Make sure your changes
    have the desired effect and no undesirable side-effects.
 5. When you are satisfied with the changes, push them to try to ensure that the
    modified tasks work as expected.
@@ -112,13 +112,109 @@ configuration is landed!  For many cases
 try: push the configuration to run the test to try along with your work to fix
 the remaining test failures.
 When working with a group, check out a "twig" repository to share among your
 group, and land the test configuration in that repository.  Once the test is
 green, merge to an integration branch and the test will begin running there as
+Adding a New Task
+If you are adding a new task that is not a test suite, there are a number of
+options.  A few questions to consider:
+ * Is this a new build platform or variant that will produce an artifact to
+   be run through the usual test suites?
+ * Does this task depend on other tasks?  Do other tasks depend on it?
+ * Is this one of a few related tasks, or will you need to generate a large
+   set of tasks using some programmatic means (for example, chunking)?
+ * How is the task actually excuted?  Mozharness?  Mach?
+ * What kind of environment does the task require?
+Armed with that information, you can choose among a few options for
+implementing this new task.  Try to choose the simplest solution that will
+satisfy your near-term needs.  Since this is all implemented in-tree, it
+is not difficult to refactor later when you need more generality.
+Existing Kind
+The simplest option is to add your task to an existing kind.  This is most
+practical when the task "makes sense" as part of that kind -- for example, if
+your task is building an installer for a new platform using mozharness scripts
+similar to the existing build tasks, it makes most sense to add your task to
+the ``build`` kind.  If you need some additional functionality in the kind,
+it's OK to modify the implementation as necessary, as long as the modification
+is complete and useful to the next developer to come along.
+New Kind
+The next option to consider is adding a new kind.  A distinct kind gives you
+some isolation from other task types, which can be nice if you are adding an
+experimental kind of task.
+Kinds can range in complexity.  The simplest sort of kind uses the
+``TransformTask`` implementation to read a list of jobs from the ``jobs`` key,
+and applies the standard ``job`` and ``task`` transforms:
+.. code-block:: yaml
+    implementation: taskgraph.task.transform:TransformTask
+    transforms:
+       - taskgraph.transforms.job:transforms
+       - taskgraph.transforms.task:transforms
+    jobs:
+       - ..your job description here..
+Custom Kind Implementation
+If your task depends on other tasks, then the decision of which tasks to create
+may require some code.  For example, the ``upload-symbols`` kind iterates over
+the builds in the graph, generating a task for each one.  This specific
+post-build behavior is implemented in the general
+``taskgraph.task.post_build:PostBuildTask`` kind implementation.  If your task
+needs something more purpose-specific, then it may be time to write a new kind
+Custom Transforms
+If your task needs to create many tasks from a single description, for example
+to implement chunking, it is time to implement some custom transforms.  Ideally
+those transforms will produce job descriptions, so you can use the existing ``job``
+and ``task`` transforms:
+.. code-block:: yaml
+    transforms:
+       - taskgraph.transforms.my_stuff:transforms
+       - taskgraph.transforms.job:transforms
+       - taskgraph.transforms.task:transforms
+Similarly, if you need to include dynamic task defaults -- perhaps some feature
+is only available in level-3 repositories, or on specific projects -- then
+custom transforms are the appropriate tool.  Try to keep transforms simple,
+single-purpose and well-documented!
+Custom Run-Using
+If the way your task is executed is unique (so, not a mach command or
+mozharness invocation), you can add a new implementation of the job
+description's "run" section.  Before you do this, consider that it might be a
+better investment to modify your task to support invocation via mozharness or
+mach, instead.  If this is not possible, then adding a new file in
+``taskcluster/taskgraph/transforms/jobs`` with a structure similar to its peers
+will make the new run-using option available for job descriptions.
 Something Else?
 If you make another change not described here that turns out to be simple or
 common, please include an update to this file in your patch.