Bug 1541041 Page timeouts are ignored during cold page-load test suites r=rwood,davehunt
authorFlorin Strugariu <fstrugariu@mozilla.com>
Wed, 08 May 2019 18:29:56 +0000
changeset 532017 921e77da2c82a59ba12cd7bca42fc39a824c31e8
parent 532016 c78bb9fab197cb3120ca2704c72b2edf1ddccb49
child 532018 a93ed2a80220b9ba15fb9b59ea79b5f1ee8f7693
push id11265
push userffxbld-merge
push dateMon, 13 May 2019 10:53:39 +0000
treeherdermozilla-beta@77e0fe8dbdd3 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersrwood, davehunt
bugs1541041
milestone68.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1541041 Page timeouts are ignored during cold page-load test suites r=rwood,davehunt Differential Revision: https://phabricator.services.mozilla.com/D26309
testing/raptor/raptor/raptor.py
--- a/testing/raptor/raptor/raptor.py
+++ b/testing/raptor/raptor/raptor.py
@@ -329,17 +329,18 @@ class Raptor(object):
             self.gecko_profiler = GeckoProfile(upload_dir,
                                                self.config,
                                                test)
 
     def wait_for_test_finish(self, test, timeout):
         # convert timeout to seconds and account for page cycles
         timeout = int(timeout / 1000) * int(test.get('page_cycles', 1))
         # account for the pause the raptor webext runner takes after browser startup
-        timeout += (int(self.post_startup_delay / 1000) + 3)
+        # and the time an exception is propagated through the framework
+        timeout += (int(self.post_startup_delay / 1000) + 10)
 
         # if geckoProfile enabled, give browser more time for profiling
         if self.config['gecko_profile'] is True:
             timeout += 5 * 60
 
         elapsed_time = 0
         while not self.control_server._finished:
             if self.config['enable_control_server_wait']:
@@ -860,16 +861,20 @@ class RaptorAndroid(Raptor):
 
             self.wait_for_test_finish(test, timeout)
 
             # in debug mode, and running locally, leave the browser running
             if self.debug_mode and self.config['run_local']:
                 self.log.info("* debug-mode enabled - please shutdown the browser manually...")
                 self.runner.wait(timeout=None)
 
+            # break test execution if a exception is present
+            if len(self.results_handler.page_timeout_list) > 0:
+                break
+
     def run_test_warm(self, test, timeout=None):
         self.log.info("test %s is running in warm mode; browser will NOT be restarted between "
                       "page cycles" % test['name'])
         if self.config['power_test']:
             init_android_power_test(self)
 
         self.run_test_setup(test)
         self.create_raptor_sdcard_folder()