Bug 1513467 - Disable raptor-tp6-amazon-firefox due to frequent failures. r=jmaher
authorCosmin Sabou <csabou@mozilla.com>
Sat, 27 Apr 2019 02:06:28 +0000
changeset 530517 873e74ad120b1d74a34328a7316736c67d0474e1
parent 530516 8b6c7d4f67eb690753179d5e6ba1635615425ae1
child 530518 dfadeda4637002db7add681209265524e1bd077e
push id11265
push userffxbld-merge
push dateMon, 13 May 2019 10:53:39 +0000
treeherdermozilla-beta@77e0fe8dbdd3 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersjmaher
bugs1513467
milestone68.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1513467 - Disable raptor-tp6-amazon-firefox due to frequent failures. r=jmaher Differential Revision: https://phabricator.services.mozilla.com/D29090
testing/raptor/raptor/tests/raptor-tp6-1.ini
testing/raptor/test/test_manifest.py
--- a/testing/raptor/raptor/tests/raptor-tp6-1.ini
+++ b/testing/raptor/raptor/tests/raptor-tp6-1.ini
@@ -19,16 +19,17 @@ gecko_profile_interval = 1
 gecko_profile_entries = 14000000
 alert_on = fcp, loadtime
 
 [raptor-tp6-amazon-firefox]
 apps = firefox
 test_url = https://www.amazon.com/s/url=search-alias%3Daps&field-keywords=laptop
 playback_recordings = amazon.mp
 measure = fnbpaint, fcp, dcf, loadtime
+disabled = Bug 1513467
 
 [raptor-tp6-facebook-firefox]
 apps = firefox
 test_url = https://www.facebook.com
 playback_recordings = facebook.mp
 measure = fnbpaint, fcp, dcf, loadtime
 
 [raptor-tp6-google-firefox]
--- a/testing/raptor/test/test_manifest.py
+++ b/testing/raptor/test/test_manifest.py
@@ -158,17 +158,17 @@ def test_validate_test_ini_valid(test_de
 def test_validate_test_ini_invalid(test_details):
     assert not (validate_test_ini(test_details))
 
 
 def test_get_raptor_test_list_firefox(create_args):
     args = create_args(browser_cycles=1)
 
     test_list = get_raptor_test_list(args, mozinfo.os)
-    assert len(test_list) == 4
+    assert len(test_list) == 3
 
     subtests = ['raptor-tp6-google-firefox', 'raptor-tp6-amazon-firefox',
                 'raptor-tp6-facebook-firefox', 'raptor-tp6-youtube-firefox']
 
     for next_subtest in test_list:
         assert next_subtest['name'] in subtests