Bug 1507908 [wpt PR 14096] - Allow stability-checker failures on Travis and report them on Taskcluster, a=testonly
☠☠ backed out by 46ad93280b06 ☠ ☠
authorPhilip Jägenstedt <philip@foolip.org>
Mon, 19 Nov 2018 18:46:28 +0000
changeset 503750 e758e29e8a1a9cfc2aea0d2fd6eace425676ec0d
parent 503749 380cb8a671e9c651b8519acd2256d1f9eeda3905
child 503751 6e08180c8ba3db7fa707e76b97069d7fc4d2f4ba
push id10290
push userffxbld-merge
push dateMon, 03 Dec 2018 16:23:23 +0000
treeherdermozilla-beta@700bed2445e6 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewerstestonly
bugs1507908, 14096, 13194
milestone65.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1507908 [wpt PR 14096] - Allow stability-checker failures on Travis and report them on Taskcluster, a=testonly Automatic update from web-platform-testsAllow stability-checker failures on Travis and report them on Taskcluster (#14096) Fixes https://github.com/web-platform-tests/wpt/issues/7660. https://github.com/web-platform-tests/wpt/issues/13194 remains open to verify that Taskcluster isn't failing to detect flakiness that Travis would. -- wpt-commits: bcd2471a16325fada550e05dad2b17895a232c6a wpt-pr: 14096
testing/web-platform/tests/.taskcluster.yml
testing/web-platform/tests/.travis.yml
--- a/testing/web-platform/tests/.taskcluster.yml
+++ b/testing/web-platform/tests/.taskcluster.yml
@@ -144,17 +144,14 @@ tasks:
                   - set -ex;
                     ~/start.sh
                       ${event.repository.clone_url}
                       refs/pull/${event.number}/merge
                       FETCH_HEAD
                       ${browser.name}
                       ${browser.channel};
                     cd ~/web-platform-tests;
-                    result=0;
                     ./tools/ci/taskcluster-run.py
                       --commit-range HEAD^
                       ${browser.name}
                       --
                       --channel=${browser.channel}
-                      ${operation.extra_args} || result=$?;
-                    echo $result > ../artifacts/run-return-code.txt;
-                    echo "Command exited with code $result (failures are allowed while this task is being vetted)."
+                      ${operation.extra_args};
--- a/testing/web-platform/tests/.travis.yml
+++ b/testing/web-platform/tests/.travis.yml
@@ -46,29 +46,27 @@ matrix:
     - name: "stability (Firefox Nightly)"
       if: type = pull_request
       os: linux
       python: "2.7"
       addons:
         apt:
           packages:
             - libnss3-tools
-      env:
-        - JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=firefox:nightly
+      env: JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=firefox:nightly
     - name: "stability (Chrome Dev)"
       if: type = pull_request
       os: linux
       python: "2.7"
       addons:
         apt:
           packages:
             - libappindicator1
             - fonts-liberation
-      env:
-        - JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=chrome:dev
+      env: JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=chrome:dev
     - name: "tools/ unittests (Python 2)"
       if: type = pull_request
       os: linux
       python: "2.7"
       env: JOB=tools_unittest TOXENV=py27 HYPOTHESIS_PROFILE=ci SCRIPT=tools/ci/ci_tools_unittest.sh
     - name: "tools/ unittests (Python 3)"
       if: type = pull_request
       os: linux
@@ -98,16 +96,18 @@ matrix:
           packages:
             - libnss3-tools
             - libappindicator1
             - fonts-liberation
   exclude:
     - env:  # exclude empty env from the top-level above
   allow_failures:
     - env: JOB=build_css SCRIPT=css/build-css-testsuites.sh
+    - env: JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=firefox:nightly
+    - env: JOB=stability SCRIPT=tools/ci/ci_stability.sh PRODUCT=chrome:dev
 script:
   - ./tools/ci/run.sh
 cache:
   directories:
     - $HOME/.cache/pip
     - $HOME/meta
 notifications:
   email: