Bug 1720292 - replace jianshu with sina r=perftest-reviewers,AlexandruIonescu
authorFlorin Strugariu <fstrugariu@mozilla.com>
Mon, 19 Jul 2021 08:22:07 +0000
changeset 585908 3b1f9f3413f4723951562044ad69bddb3b992418
parent 585907 c70dac69ebe00c4f3a7e7610da463cf9326041b6
child 585909 e7e916470a4aafc5005ae2a60bab63c56466770d
push id38623
push userabutkovits@mozilla.com
push dateMon, 19 Jul 2021 15:46:04 +0000
treeherdermozilla-central@86d0c64032ca [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersperftest-reviewers, AlexandruIonescu
bugs1720292
milestone92.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1720292 - replace jianshu with sina r=perftest-reviewers,AlexandruIonescu Differential Revision: https://phabricator.services.mozilla.com/D119762
taskcluster/ci/test/browsertime-mobile.yml
testing/perfdocs/generated/raptor.rst
testing/performance/pageload_sites.json
testing/raptor/raptor/perfdocs/config.yml
testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.ini
testing/raptor/raptor/tooltool-manifests/playback/mitm4-pixel2-fennec-jianshu.manifest
testing/raptor/raptor/tooltool-manifests/playback/mitm6-android-fenix-sina.manifest
testing/raptor/webext/raptor/manifest.json
--- a/taskcluster/ci/test/browsertime-mobile.yml
+++ b/taskcluster/ci/test/browsertime-mobile.yml
@@ -90,18 +90,18 @@ browsertime-tp6m:
         - [cnn-ampstories, cnn-amp]
         - [ebay-kleinanzeigen, ebay-k]
         - [ebay-kleinanzeigen-search, ebay-k-s]
         - [facebook-cristiano, fb-cris]
         - [google-maps, gmaps]
         - [google-search-restaurants, gsearch-r]
         - instagram
         - imdb
-        - jianshu
         - reddit
+        - sina
         - [stackoverflow, stacko]
         - web-de
         - wikipedia
         - youtube
     run-on-projects:
         by-app:
             geckoview:
                 by-test-platform:
--- a/testing/perfdocs/generated/raptor.rst
+++ b/testing/perfdocs/generated/raptor.rst
@@ -5404,98 +5404,16 @@ Page-load performance test suite on Andr
       * test-android-hw-s7-8-0-android-aarch64/opt
             * browsertime-tp6m-fenix-instagram-e10s: None
             * browsertime-tp6m-geckoview-instagram-e10s: None
             * browsertime-tp6m-live-fenix-instagram-e10s: None
             * browsertime-tp6m-live-geckoview-instagram-e10s: None
             * browsertime-tp6m-refbrow-instagram-e10s: None
 
 
-.. dropdown:: jianshu (BT, GV, FE, RB, CH-M)
-   :container: + anchor-id-jianshu-m
-
-   * **alert on**: fcp, loadtime
-   * **alert threshold**: 2.0
-   * **apps**: geckoview, fenix, refbrow, chrome-m
-   * **browser cycles**: 15
-   * **expected**: pass
-   * **lower is better**: true
-   * **page cycles**: 25
-   * **page timeout**: 60000
-   * **playback**: mitmproxy-android
-   * **playback pageset manifest**: mitm4-pixel2-fennec-jianshu.manifest
-   * **playback recordings**: mitm4-pixel2-fennec-jianshu.mp
-   * **playback version**: 6.0.2
-   * **test url**: `<https://www.jianshu.com/>`__
-   * **type**: pageload
-   * **unit**: ms
-   * **use live sites**: false
-   * **Test Task**:
-      * test-android-hw-g5-7-0-arm7-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-g5-7-0-arm7-shippable-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: mozilla-beta, trunk
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-g5-7-0-arm7-shippable/opt
-            * browsertime-tp6m-chrome-m-jianshu-e10s: None
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-g5-7-0-arm7/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-android-aarch64-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-android-aarch64-shippable-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: mozilla-beta, trunk
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-android-aarch64-shippable/opt
-            * browsertime-tp6m-chrome-m-jianshu-e10s: None
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-android-aarch64/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-arm7-shippable/opt
-            * browsertime-tp6m-chrome-m-jianshu-e10s: None
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-p2-8-0-arm7/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-s7-8-0-android-aarch64-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-s7-8-0-android-aarch64-shippable-qr/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-s7-8-0-android-aarch64-shippable/opt
-            * browsertime-tp6m-chrome-m-jianshu-e10s: None
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-      * test-android-hw-s7-8-0-android-aarch64/opt
-            * browsertime-tp6m-fenix-jianshu-e10s: None
-            * browsertime-tp6m-geckoview-jianshu-e10s: None
-            * browsertime-tp6m-refbrow-jianshu-e10s: None
-
-
 .. dropdown:: microsoft-support (BT, GV, FE, RB, CH-M)
    :container: + anchor-id-microsoft-support-m
 
    * **alert on**: fcp, loadtime
    * **alert threshold**: 2.0
    * **apps**: geckoview, fenix, refbrow, chrome-m
    * **browser cycles**: 15
    * **expected**: pass
@@ -5714,16 +5632,98 @@ Page-load performance test suite on Andr
       * test-android-hw-s7-8-0-android-aarch64/opt
             * browsertime-tp6m-fenix-reddit-e10s: None
             * browsertime-tp6m-geckoview-reddit-e10s: None
             * browsertime-tp6m-live-fenix-reddit-e10s: None
             * browsertime-tp6m-live-geckoview-reddit-e10s: None
             * browsertime-tp6m-refbrow-reddit-e10s: None
 
 
+.. dropdown:: sina (BT, GV, FE, RB, CH-M)
+   :container: + anchor-id-sina-m
+
+   * **alert on**: fcp, loadtime
+   * **alert threshold**: 2.0
+   * **apps**: geckoview, fenix, refbrow, chrome-m
+   * **browser cycles**: 15
+   * **expected**: pass
+   * **lower is better**: true
+   * **page cycles**: 25
+   * **page timeout**: 60000
+   * **playback**: mitmproxy-android
+   * **playback pageset manifest**: mitm6-android-fenix-sina.manifest
+   * **playback recordings**: mitm6-android-fenix-sina.mp
+   * **playback version**: 6.0.2
+   * **test url**: `<https://www.sina.com.cn/>`__
+   * **type**: pageload
+   * **unit**: ms
+   * **use live sites**: false
+   * **Test Task**:
+      * test-android-hw-g5-7-0-arm7-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-g5-7-0-arm7-shippable-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: mozilla-beta, trunk
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-g5-7-0-arm7-shippable/opt
+            * browsertime-tp6m-chrome-m-sina-e10s: None
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-g5-7-0-arm7/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-android-aarch64-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-android-aarch64-shippable-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: mozilla-beta, trunk
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-android-aarch64-shippable/opt
+            * browsertime-tp6m-chrome-m-sina-e10s: None
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-android-aarch64/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-arm7-shippable/opt
+            * browsertime-tp6m-chrome-m-sina-e10s: None
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-p2-8-0-arm7/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-s7-8-0-android-aarch64-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-s7-8-0-android-aarch64-shippable-qr/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-s7-8-0-android-aarch64-shippable/opt
+            * browsertime-tp6m-chrome-m-sina-e10s: None
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+      * test-android-hw-s7-8-0-android-aarch64/opt
+            * browsertime-tp6m-fenix-sina-e10s: None
+            * browsertime-tp6m-geckoview-sina-e10s: None
+            * browsertime-tp6m-refbrow-sina-e10s: None
+
+
 .. dropdown:: stackoverflow (BT, GV, FE, RB, CH-M)
    :container: + anchor-id-stackoverflow-m
 
    * **alert on**: fcp, loadtime
    * **alert threshold**: 2.0
    * **apps**: geckoview, fenix, refbrow, chrome-m
    * **browser cycles**: 15
    * **expected**: pass
--- a/testing/performance/pageload_sites.json
+++ b/testing/performance/pageload_sites.json
@@ -92,20 +92,19 @@
       "test_url": "https://m.imdb.com/"
     },
     {
       "login": true,
       "name": "instagram",
       "test_url": "https://www.instagram.com"
     },
     {
-      "login": true,
-      "comment": "not login test but fails to load properly",
-      "name": "jianshu",
-      "test_url": "https://www.jianshu.com/"
+      "login": false,
+      "name": "sina",
+      "test_url": "https://www.sina.com.cn/"
     },
     {
       "login": false,
       "name": "microsoft-support",
       "test_url": "https://support.microsoft.com/en-us"
     },
     {
       "login": false,
--- a/testing/raptor/raptor/perfdocs/config.yml
+++ b/testing/raptor/raptor/perfdocs/config.yml
@@ -59,20 +59,20 @@ suites:
             espn: "BT, GV, FE, RB, CH-M"
             facebook: "BT, GV, FE, RB, CH-M"
             facebook-cristiano: "BT, GV, FE, RB, CH-M"
             google: "BT, GV, FE, RB, CH-M"
             google-maps: "BT, GV, FE, RB, CH-M"
             google-search-restaurants: "BT, GV, FE, RB, CH-M"
             imdb: "BT, GV, FE, RB, CH-M"
             instagram: "BT, GV, FE, RB, CH-M"
-            jianshu: "BT, GV, FE, RB, CH-M"
             microsoft-support: "BT, GV, FE, RB, CH-M"
             reddit: "BT, GV, FE, RB, CH-M"
             stackoverflow: "BT, GV, FE, RB, CH-M"
+            sina: "BT, GV, FE, RB, CH-M"
             web-de: "BT, GV, FE, RB, CH-M"
             wikipedia: "BT, GV, FE, RB, CH-M"
             youtube-watch: "BT, GV, FE, RB, CH-M"
     live:
         description: "A set of test pages that are run as live sites instead of recorded versions. These tests are available on all browsers, on all platforms."
     benchmarks:
         description: "Standard benchmarks are third-party tests (i.e. Speedometer) that we have integrated into Raptor to run per-commit in our production CI. "
     scenario:
--- a/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.ini
+++ b/testing/raptor/raptor/tests/tp6/mobile/browsertime-tp6m.ini
@@ -90,27 +90,25 @@ test_url = https://m.imdb.com/
 
 [instagram]
 test_url = https://www.instagram.com
 playback_pageset_manifest = mitm6-g5-fenix-{subtest}.manifest
 playback_recordings = mitm6-g5-fenix-{subtest}.mp
 playback_version = 6.0.2
 login = true
 
-[jianshu]
-test_url = https://www.jianshu.com/
-playback_pageset_manifest = mitm4-pixel2-fennec-{subtest}.manifest
-playback_recordings = mitm4-pixel2-fennec-{subtest}.mp
-
 [microsoft-support]
 test_url = https://support.microsoft.com/en-us
 
 [reddit]
 test_url = https://www.reddit.com
 
+[sina]
+test_url = https://www.sina.com.cn/
+
 [stackoverflow]
 test_url = https://stackoverflow.com/
 
 [web-de]
 test_url = https://web.de/magazine/politik/politologe-glaubt-grossen-koalition-herbst-knallen-33563566
 
 [wikipedia]
 test_url = https://en.m.wikipedia.org/wiki/Main_Page
deleted file mode 100644
--- a/testing/raptor/raptor/tooltool-manifests/playback/mitm4-pixel2-fennec-jianshu.manifest
+++ /dev/null
@@ -1,9 +0,0 @@
-[
-  {
-    "algorithm": "sha512",
-    "digest": "83c6c843cf03c98db5fceb11051643b955c881c1226578697f37b5066f889ac3c090dfd86d298bd11391b47d904ace59b17f7d0fbe4249b8249bf317a2721a90",
-    "filename": "mitm4-pixel2-fennec-jianshu.zip",
-    "size": 10055373,
-    "visibility": "public"
-  }
-]
new file mode 100644
--- /dev/null
+++ b/testing/raptor/raptor/tooltool-manifests/playback/mitm6-android-fenix-sina.manifest
@@ -0,0 +1,9 @@
+[
+  {
+    "algorithm": "sha512",
+    "digest": "f3b55b61e6c38a342fee68fef04196de39b209053348cf58f5d9cc289100d82f271008f065adfd4ec02df564f6de12726f198e151af24aace9f654bf008a7b86",
+    "filename": "mitm6-android-fenix-sina.zip",
+    "size": 14757830,
+    "visibility": "public"
+  }
+]
--- a/testing/raptor/webext/raptor/manifest.json
+++ b/testing/raptor/webext/raptor/manifest.json
@@ -25,26 +25,26 @@
                   "*://*.ebay-kleinanzeigen.de/*",
                   "*://*.espn.com/*",
                   "*://*.facebook.com/*",
                   "*://*.fandom.com/*",
                   "*://*.google.com/*",
                   "*://*.imdb.com/*",
                   "*://*.imgur.com/*",
                   "*://*.instagram.com/*",
-                  "*://*.jianshu.com/*",
                   "*://*.linkedin.com/*",
                   "*://*.live.com/*",
                   "*://*.microsoft.com/*",
                   "*://*.netflix.com/*",
                   "*://*.office.com/*",
                   "*://*.paypal.com/*",
                   "*://*.pinterest.com/*",
                   "*://*.reddit.com/*",
                   "*://*.stackoverflow.com/*",
+                  "*://*.sina.com.cn/*",
                   "*://*.tumblr.com/*",
                   "*://*.twitch.tv/*",
                   "*://*.twitter.com/*",
                   "*://*.vice.com/*",
                   "*://*.web.de/*",
                   "*://*.wikia.com/*",
                   "*://*.wikipedia.org/*",
                   "*://*.yahoo.com/*",