Bug 1531360 - [fuzzy] Add ability to intersect queries, r=jgraham
authorAndrew Halberstadt <ahalberstadt@mozilla.com>
Wed, 06 Mar 2019 16:14:00 +0000
changeset 462697 cd1704c276dfc0eab6552d29bdae3d01bf656774
parent 462696 b5d943805f64c1b3dd97f3fbb31f5a0acfe465b7
child 462698 c66813164419e99c5e8d0343a0386a0251550455
push id79794
push userahalberstadt@mozilla.com
push dateWed, 06 Mar 2019 19:54:45 +0000
treeherderautoland@cd1704c276df [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersjgraham
bugs1531360
milestone67.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1531360 - [fuzzy] Add ability to intersect queries, r=jgraham Differential Revision: https://phabricator.services.mozilla.com/D22028
tools/tryselect/docs/selectors/fuzzy.rst
tools/tryselect/selectors/fuzzy.py
tools/tryselect/test/test_fuzzy.t
tools/tryselect/test/test_preset.t
--- a/tools/tryselect/docs/selectors/fuzzy.rst
+++ b/tools/tryselect/docs/selectors/fuzzy.rst
@@ -63,16 +63,51 @@ Would likely match a similar set of task
     'a | 'b  => OR operator (joins two exact match operators together)
 
 For example:
 
 .. code-block:: text
 
     ^start 'exact | !ignore fuzzy end$
 
+Specifying Queries on the Command Line
+--------------------------------------
+
+Sometimes it's more convenient to skip the interactive interface and specify a query on the command
+line with ``-q/--query``. This is equivalent to opening the interface then typing:
+``<query><ctrl-a><enter>``.
+
+For example:
+
+.. code-block:: shell
+
+    # selects all mochitest tasks
+    $ mach try fuzzy --query "mochitest"
+
+You can pass in multiple queries at once and the results of each will be joined together:
+
+.. code-block:: shell
+
+    # selects all mochitest and reftest tasks
+    $ mach try fuzzy -q "mochitest" -q "reftest"
+
+If instead you want the intersection of queries, you can pass in ``-x/--and``:
+
+.. code-block:: shell
+
+    # selects all windows mochitest tasks
+    $ mach try fuzzy --and -q "mochitest" -q "windows"
+
+Using query intersections is especially useful with presets:
+
+.. code-block:: shell
+
+    # selects all windows perf tasks
+    $ mach try fuzzy --preset perf -xq "windows"
+
 Test Paths
 ----------
 
 One or more paths to a file or directory may be specified as positional arguments. When
 specifying paths, the list of available tasks to choose from is filtered down such that
 only suites that have tests in a specified path can be selected. Notably, only the first
 chunk of each suite/platform appears. When the tasks are scheduled, only tests that live
 under one of the specified paths will be run.
@@ -116,19 +151,20 @@ Would produce the following ``try_task_c
 Inside of these tasks, the reftest harness will only run tests that live under
 ``layout/reftests/reftest-sanity``.
 
 Additional Arguments
 --------------------
 
 There are a few additional command line arguments you may wish to use:
 
-``-q/--query``
-Instead of opening the interactive interface, automatically apply the specified
-query. This is equivalent to opening the interface then typing: ``<query><ctrl-a><enter>``.
+``-e/--exact``
+By default, fzf treats terms as a fuzzy match and prefixing a term with ``'`` turns it into an exact
+match. If passing in ``--exact``, this behaviour is reversed. Non-prefixed terms become exact, and a
+``'`` prefix makes a term fuzzy.
 
 ``--full``
 By default, only target tasks (e.g tasks that would normally run on mozilla-central)
 are generated. Passing in ``--full`` allows you to select from all tasks. This is useful for
 things like nightly or release tasks.
 
 ``-u/--update``
 Update the bootstrapped fzf binary to the latest version.
--- a/tools/tryselect/selectors/fuzzy.py
+++ b/tools/tryselect/selectors/fuzzy.py
@@ -87,16 +87,24 @@ class FuzzyParser(BaseTryParser):
         [['-q', '--query'],
          {'metavar': 'STR',
           'action': 'append',
           'help': "Use the given query instead of entering the selection "
                   "interface. Equivalent to typing <query><ctrl-a><enter> "
                   "from the interface. Specifying multiple times schedules "
                   "the union of computed tasks.",
           }],
+        [['-x', '--and'],
+         {'dest': 'intersection',
+          'action': 'store_true',
+          'default': False,
+          'help': "When specifying queries on the command line with -q/--query, "
+                  "use the intersection of tasks rather than the union. This is "
+                  "especially useful for post filtering presets.",
+          }],
         [['-e', '--exact'],
          {'action': 'store_true',
           'default': False,
           'help': "Enable exact match mode. Terms will use an exact match "
                   "by default, and terms prefixed with ' will become fuzzy."
           }],
         [['-u', '--update'],
          {'action': 'store_true',
@@ -199,17 +207,18 @@ def run_fzf(cmd, tasks):
     return query, selected
 
 
 def filter_target_task(task):
     return not any(re.search(pattern, task) for pattern in TARGET_TASK_FILTERS)
 
 
 def run(update=False, query=None, templates=None, full=False, parameters=None, save_query=False,
-        push=True, message='{msg}', paths=None, exact=False, closed_tree=False):
+        push=True, message='{msg}', paths=None, exact=False, closed_tree=False,
+        intersection=False):
     fzf = fzf_bootstrap(update)
 
     if not fzf:
         print(FZF_NOT_FOUND)
         return 1
 
     check_working_directory(push)
     tg = generate_tasks(parameters, full, root=vcs.path)
@@ -249,18 +258,23 @@ def run(update=False, query=None, templa
     else:
         commands.append(base_cmd)
 
     queries = []
     selected = set()
     for command in commands:
         query, tasks = run_fzf(command, all_tasks)
         if tasks:
+            tasks = set(tasks)
             queries.append(query)
-            selected.update(tasks)
+
+            if intersection and selected:
+                selected &= tasks
+            else:
+                selected |= tasks
 
     if not selected:
         print("no tasks selected")
         return
 
     if save_query:
         return queries
 
--- a/tools/tryselect/test/test_fuzzy.t
+++ b/tools/tryselect/test/test_fuzzy.t
@@ -63,16 +63,37 @@ Test multiple selectors
           "env": {
               "TRY_SELECTOR": "fuzzy"
           }
       },
       "version": 1
   }
   
 
+Test query intersection
+
+  $ ./mach try fuzzy $testargs --and -q "'foo" -q "'opt"
+  Commit message:
+  Fuzzy query='foo&query='opt
+  
+  Pushed via `mach try fuzzy`
+  Calculated try_task_config.json:
+  {
+      "tasks": [
+          "test/foo-opt"
+      ],
+      "templates": {
+          "env": {
+              "TRY_SELECTOR": "fuzzy"
+          }
+      },
+      "version": 1
+  }
+  
+
 
 Test exact match
 
   $ ./mach try fuzzy $testargs --full -q "testfoo | 'testbar"
   Commit message:
   Fuzzy query=testfoo | 'testbar
   
   Pushed via `mach try fuzzy`
--- a/tools/tryselect/test/test_preset.t
+++ b/tools/tryselect/test/test_preset.t
@@ -118,16 +118,17 @@ Test preset with syntax subcommand
     - none
     tests:
     - mochitests
 
 Test preset with fuzzy subcommand
 
   $ ./mach try fuzzy $testargs --save baz -q "'foo" --rebuild 5
   preset saved, run with: --preset=baz
+
   $ ./mach try fuzzy $testargs --preset baz
   Commit message:
   Fuzzy query='foo
   
   Pushed via `mach try fuzzy`
   Calculated try_task_config.json:
   {
       "tasks": [
@@ -183,16 +184,37 @@ Queries can be appended to presets
           "env": {
               "TRY_SELECTOR": "fuzzy"
           },
           "rebuild": 5
       },
       "version": 1
   }
   
+
+  $ ./mach try $testargs --preset baz -xq "'opt"
+  Commit message:
+  Fuzzy query='foo&query='opt
+  
+  Pushed via `mach try fuzzy`
+  Calculated try_task_config.json:
+  {
+      "tasks": [
+          "test/foo-opt"
+      ],
+      "templates": {
+          "env": {
+              "TRY_SELECTOR": "fuzzy"
+          },
+          "rebuild": 5
+      },
+      "version": 1
+  }
+  
+
   $ ./mach try fuzzy $testargs --list-presets
   Presets from */mozbuild/try_presets.yml: (glob)
   
     bar:
       no_artifact: true
       platforms:
       - win32
       push: false