Bug 1486514 [wpt PR 12695] - Make reftests 800x600 everywhere, a=testonly
authorSam Sneddon <me@gsnedders.com>
Tue, 05 Mar 2019 12:19:18 +0000
changeset 525487 ed02da4c6be28b064d9c6168f54c7f84036fa9e1
parent 525486 fee781812ccfff0533cee83a689d9ba8eb8819a7
child 525488 578acd858f710f5cc9d2fccb20c7513bcc125ad0
push id2032
push userffxbld-merge
push dateMon, 13 May 2019 09:36:57 +0000
treeherdermozilla-release@455c1065dcbe [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewerstestonly
bugs1486514, 12695
milestone67.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1486514 [wpt PR 12695] - Make reftests 800x600 everywhere, a=testonly Automatic update from web-platform-tests Change reftest size to 800x600 everywhere (#12695) With Firefox, with the internal reftest runner (used by default), this doesn't actually change the reftest size in Firefox versions prior to 66 (currently on the beta channel); to get the correct behaviour with older versions of Firefox, you must pass `--reftest-external` to use the external reftest runner. -- wpt-commits: d5f07ed3b5186e553940a3d5c547067850a4e3f0 wpt-pr: 12695
testing/web-platform/tests/docs/_reviewing-tests/checklist.md
testing/web-platform/tests/docs/_writing-tests/assumptions.md
testing/web-platform/tests/docs/_writing-tests/general-guidelines.md
testing/web-platform/tests/docs/_writing-tests/reftests.md
testing/web-platform/tests/infrastructure/reftest/size-ref.html
testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/fennec.py
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorselenium.py
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorservo.py
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorwebdriver.py
testing/web-platform/tests/tools/wptrunner/wptrunner/executors/reftest.js
--- a/testing/web-platform/tests/docs/_reviewing-tests/checklist.md
+++ b/testing/web-platform/tests/docs/_reviewing-tests/checklist.md
@@ -105,17 +105,17 @@ identically to the test on all platforms
 <label>
 <input type="checkbox">
 The reference file uses a different technique that won't fail in
 the same way as the test.
 </label>
 
 <label>
 <input type="checkbox">
-The test and reference render within a 600x600 viewport, only displaying
+The test and reference render within a 800x600 viewport, only displaying
 scrollbars if their presence is being tested.
 </label>
 
 <label>
 <input type="checkbox">
 **Nit**: The test has a self-describing statement.
 </label>
 
@@ -175,17 +175,17 @@ The test has a self-describing statement
 The self-describing statement is accurate, precise, simple, and
 self-explanatory. Someone with no technical knowledge should be able to say
 whether the test passed or failed within a few seconds, and not need to spend
 several minutes thinking or asking questions.
 </label>
 
 <label>
 <input type="checkbox">
-The test renders within a 600x600 viewport, only displaying scrollbars if their
+The test renders within a 800x600 viewport, only displaying scrollbars if their
 presence is being tested.
 </label>
 
 <label>
 <input type="checkbox">
 The test renders to a fixed, static page with no animation.
 </label>
 
--- a/testing/web-platform/tests/docs/_writing-tests/assumptions.md
+++ b/testing/web-platform/tests/docs/_writing-tests/assumptions.md
@@ -3,17 +3,17 @@ layout: page
 title: Test Assumptions
 order: 13
 ---
 
 The tests make a number of assumptions of the user agent, and new
 tests can freely rely on these assumptions being true:
 
  * The device is a full-color device.
- * The device has a viewport width of at least 600px.
+ * The device has a viewport width of at least 800px.
  * The UA imposes no minimum font size.
  * The `medium` `font-size` computes to 16px.
  * The canvas background is `white`.
  * The initial value of `color` is `black`.
  * The user stylesheet is empty (except where indicated by the tests).
  * The device is interactive and uses scroll bars.
  * The device has the Ahem font installed.
  * The HTML `div` element is assigned `display: block;`, the
--- a/testing/web-platform/tests/docs/_writing-tests/general-guidelines.md
+++ b/testing/web-platform/tests/docs/_writing-tests/general-guidelines.md
@@ -85,17 +85,17 @@ supports [a variety of features][server 
 browsers, including (but not limited to!) support for writing out
 appropriate domains and custom (per-file and per-directory) HTTP
 headers.
 
 
 ### Be Short
 
 Tests should be as short as possible. For reftests in particular
-scrollbars at 600&#xD7;600px window size must be avoided unless scrolling
+scrollbars at 800&#xD7;600px window size must be avoided unless scrolling
 behavior is specifically being tested. For all tests extraneous
 elements on the page should be avoided so it is clear what is part of
 the test (for a typical testharness test, the only content on the page
 will be rendered by the harness itself).
 
 
 ### Be Minimal
 
--- a/testing/web-platform/tests/docs/_writing-tests/reftests.md
+++ b/testing/web-platform/tests/docs/_writing-tests/reftests.md
@@ -22,17 +22,17 @@ cause the test to fail.
 In the simplest case, a reftest consists of a pair of files called the
 *test* and the *reference*.
 
 The *test* file is the one that makes use of the technology being
 tested. It also contains a `link` element with `rel="match"` or
 `rel="mismatch"` and `href` attribute pointing to the *reference*
 file, e.g. `<link rel=match href=references/green-box-ref.html>`. A
 `match` test only passes if the two files render pixel-for-pixel
-identically within a 600x600 window *including* scroll-bars if
+identically within a 800x600 window *including* scroll-bars if
 present; a `mismatch` test only passes if they *don't* render
 identically.
 
 The *reference* file is typically written to be as simple as possible,
 and does not use the technology under test. It is desirable that the
 reference be rendered correctly even in UAs with relatively poor
 support for CSS and no support for the technology under test.
 
--- a/testing/web-platform/tests/infrastructure/reftest/size-ref.html
+++ b/testing/web-platform/tests/infrastructure/reftest/size-ref.html
@@ -1,2 +1,2 @@
 <!doctype html>
-<p>innerWidth x innerHeight: <span>600x600</span></p>
+<p>innerWidth x innerHeight: <span>800x600</span></p>
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/fennec.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/browsers/fennec.py
@@ -141,17 +141,17 @@ class FennecBrowser(FirefoxBrowser):
         self.profile = FirefoxProfile(preferences=preferences)
         self.profile.set_preferences({"marionette.port": self.marionette_port,
                                       "dom.disable_open_during_load": False,
                                       "places.history.enabled": False,
                                       "dom.send_after_paint_to_content": True,
                                       "network.preload": True})
         if self.test_type == "reftest":
             self.logger.info("Setting android reftest preferences")
-            self.profile.set_preferences({"browser.viewport.desktopWidth": 600,
+            self.profile.set_preferences({"browser.viewport.desktopWidth": 800,
                                           # Disable high DPI
                                           "layout.css.devPixelsPerPx": "1.0",
                                           # Ensure that the full browser element
                                           # appears in the screenshot
                                           "apz.allow_zooming": False,
                                           "android.widget_paints_background": False,
                                           # Ensure that scrollbars are always painted
                                           "layout.testing.overlay-scrollbars.always-visible": True})
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py
@@ -863,17 +863,19 @@ class InternalRefTestImplementation(obje
 
     def run_test(self, test):
         references = self.get_references(test)
         timeout = (test.timeout * 1000) * self.timeout_multiplier
         rv = self.executor.protocol.marionette._send_message("reftest:run",
                                                              {"test": self.executor.test_url(test),
                                                               "references": references,
                                                               "expected": test.expected(),
-                                                              "timeout": timeout})["value"]
+                                                              "timeout": timeout,
+                                                              "width": 800,
+                                                              "height": 600})["value"]
         return rv
 
     def get_references(self, node):
         rv = []
         for item, relation in node.references:
             rv.append([self.executor.test_url(item), self.get_references(item), relation])
         return rv
 
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorselenium.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorselenium.py
@@ -367,17 +367,17 @@ class SeleniumRefTestExecutor(RefTestExe
 
     def do_test(self, test):
         self.logger.info("Test requires OS-level window focus")
 
         width_offset, height_offset = self.protocol.webdriver.execute_script(
             """return [window.outerWidth - window.innerWidth,
                        window.outerHeight - window.innerHeight];"""
         )
-        self.protocol.webdriver.set_window_rect(0, 0, 600 + width_offset, 600 + height_offset)
+        self.protocol.webdriver.set_window_rect(0, 0, 800 + width_offset, 600 + height_offset)
 
         result = self.implementation.run_test(test)
 
         return self.convert_result(test, result)
 
     def screenshot(self, test, viewport_size, dpi):
         # https://github.com/w3c/wptrunner/issues/166
         assert viewport_size is None
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorservo.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorservo.py
@@ -206,17 +206,17 @@ class ServoRefTestExecutor(ProcessTestEx
                 self.debug_info)
 
             for stylesheet in self.browser.user_stylesheets:
                 command += ["--user-stylesheet", stylesheet]
 
             for pref, value in test.environment.get('prefs', {}).iteritems():
                 command += ["--pref", "%s=%s" % (pref, value)]
 
-            command += ["--resolution", viewport_size or "600x600"]
+            command += ["--resolution", viewport_size or "800x600"]
 
             if self.browser.ca_certificate_path:
                 command += ["--certificate-path", self.browser.ca_certificate_path]
 
             if dpi:
                 command += ["--device-pixel-ratio", dpi]
 
             # Run ref tests in headless mode
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorwebdriver.py
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executorwebdriver.py
@@ -401,17 +401,17 @@ class WebDriverRefTestExecutor(RefTestEx
         return self.protocol.is_alive()
 
     def do_test(self, test):
         width_offset, height_offset = self.protocol.webdriver.execute_script(
             """return [window.outerWidth - window.innerWidth,
                        window.outerHeight - window.innerHeight];"""
         )
         self.protocol.webdriver.window.position = (0, 0)
-        self.protocol.webdriver.window.size = (600 + width_offset, 600 + height_offset)
+        self.protocol.webdriver.window.size = (800 + width_offset, 600 + height_offset)
 
         result = self.implementation.run_test(test)
 
         return self.convert_result(test, result)
 
     def screenshot(self, test, viewport_size, dpi):
         # https://github.com/w3c/wptrunner/issues/166
         assert viewport_size is None
--- a/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/reftest.js
+++ b/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/reftest.js
@@ -1,1 +1,1 @@
-var win = window.open("about:blank", "test", "left=0,top=0,width=600,height=600");
+var win = window.open("about:blank", "test", "left=0,top=0,width=800,height=600");