Bug 1597180 - Add build and prefs config to enable Thunderbird telemetry reporting. r=mkmelin DONTBUILD
authorBen Campbell <benc@thunderbird.net>
Tue, 14 Jan 2020 12:41:05 +0200
changeset 37075 f8a6cf1ee08f8223c36bb92414e29734d8d6c9d9
parent 37074 2a388e5064a108d031c048b3004c2a77e80caf0f
child 37076 1784150a888a0a5d9e9416a3fee9926fc7907324
push id2552
push userclokep@gmail.com
push dateMon, 10 Feb 2020 21:24:16 +0000
treeherdercomm-beta@f95a6f4408a3 [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersmkmelin
bugs1597180
Bug 1597180 - Add build and prefs config to enable Thunderbird telemetry reporting. r=mkmelin DONTBUILD
mail/app/profile/all-thunderbird.js
mail/components/telemetry/README.md
--- a/mail/app/profile/all-thunderbird.js
+++ b/mail/app/profile/all-thunderbird.js
@@ -364,17 +364,16 @@ pref("security.warn_viewing_mixed", fals
 pref("security.aboutcertificate.enabled", false);
 
 pref("general.config.obscure_value", 0); // for MCD .cfg files
 
 pref("browser.display.auto_quality_min_font_size", 0);
 
 pref("view_source.syntax_highlight", false);
 
-pref("toolkit.telemetry.infoURL", "https://www.mozilla.org/thunderbird/legal/privacy/#telemetry");
 
 /////////////////////////////////////////////////////////////////
 // End core all.js pref overrides
 /////////////////////////////////////////////////////////////////
 
 /////////////////////////////////////////////////////////////////
 // Generic browser related prefs.
 /////////////////////////////////////////////////////////////////
@@ -1225,8 +1224,44 @@ pref("devtools.popup.disable_autohide", 
 pref("devtools.toolbox.content-frame", true);
 
 // Visibility switch preference for the WhatsNew panel.
 pref("devtools.whatsnew.enabled", true);
 
 // Temporary preference to fully disable the WhatsNew panel on any target.
 // Should be removed in https://bugzilla.mozilla.org/show_bug.cgi?id=1596037
 pref("devtools.whatsnew.feature-enabled", true);
+
+// Telemetry settings.
+
+// Server to submit telemetry pings to.
+// For now we'll just default to local testing:
+pref("toolkit.telemetry.server", "http://localhost:8080");
+pref("toolkit.telemetry.server_owner", "Thunderbird");
+
+// Determines if Telemetry pings can be archived locally.
+pref("toolkit.telemetry.archive.enabled", true);
+// Enables sending the shutdown ping when Thunderbird shuts down.
+pref("toolkit.telemetry.shutdownPingSender.enabled", true);
+// Enables sending the shutdown ping using the pingsender from the first session.
+pref("toolkit.telemetry.shutdownPingSender.enabledFirstSession", false);
+// Enables sending a duplicate of the first shutdown ping from the first session.
+pref("toolkit.telemetry.firstShutdownPing.enabled", true);
+// Enables sending the 'new-profile' ping on new profiles.
+pref("toolkit.telemetry.newProfilePing.enabled", true);
+// Enables sending 'update' pings on Thunderbird updates.
+pref("toolkit.telemetry.updatePing.enabled", true);
+// Enables sending 'bhr' pings when the app hangs.
+pref("toolkit.telemetry.bhrPing.enabled", true);
+// Whether to enable Ecosystem Telemetry, requires a restart.
+#ifdef NIGHTLY_BUILD
+  pref("toolkit.telemetry.ecosystemtelemetry.enabled", true);
+#else
+  pref("toolkit.telemetry.ecosystemtelemetry.enabled", false);
+#endif
+
+// Required to enable telemetry pings (defaults to true if
+// MOZ_SERVICES_HEALTHREPORT is defined, but we're not yet sure we want
+// that...)
+pref(`datareporting.healthreport.uploadEnabled`, true);
+
+pref("toolkit.telemetry.infoURL", "https://www.mozilla.org/thunderbird/legal/privacy/#telemetry");
+
--- a/mail/components/telemetry/README.md
+++ b/mail/components/telemetry/README.md
@@ -30,94 +30,143 @@ about slapping on prefixes to our defini
 For Events and Scalars, we keep everything under `tb.`.
 
 For Histograms, we use a `TB_` or `TELEMETRY_TEST_TB_` prefix.
 
 (Why not just `TB_`? Because the telemetry test helper functions
 `getSnapshotForHistograms()`/`getSnapshotForKeyedHistograms()` have an option
 to filter out histograms with a `TELEMETRY_TEST_` prefix).
 
-
-## Enabling telemetry
-
-### Compile-time
+## Compile-time switches
 
 Telemetry is not compiled in by default. You need to add the following line
 to your mozconfig:
 
     export MOZ_TELEMETRY_REPORTING=1
 
 The nightly and release configs have this setting already (`$ grep -r MOZ_TELEMETRY_ mail/config/mozconfigs`).
 
+## Runtime prefs for testing
 
-### At run time
+There are a few `user.js` settings you'll want to set up for enabling telemetry local builds:
+
+### Send telemetry to a local server
 
-There's a complex set of conditions to enable telemetry reporting.
-The runtime settings needed for a minimal test setup are:
+You'll want to set the telemetry end point to a locally-running http server, eg:
+```
+user_pref("toolkit.telemetry.server", "http://localhost:12345");
+user_pref("toolkit.telemetry.server_owner", "TimmyTestfish");
+user_pref("datareporting.healthreport.uploadEnabled",true);
+```
+
+For a simple test server, try https://github.com/mozilla/gzipServer
+(or alternatively https://github.com/bcampbell/webhole).
+
+### Override the official-build-only check
 
-- `toolkit.telemetry.server` - URL where the collected data will be POSTed to
-   (`https://incoming.telemetry.mozilla.org`). So if you're running a local
-   server for testing, you'll likely want this to be some localhost URL.
-- `toolkit.telemetry.server.owner` - The owner of the server (`Mozilla`).
-   The implication is that it's polite to change this if you're running a
-   non-Mozilla server.
-- `toolkit.telemetry.send.overrideOfficialCheck` - usually, telemetry is only
-   send for official builds (ie `export MOZILLA_OFFICIAL=1` in `mozconfig`).
-   Setting this to `true` enables sending for unofficial builds.
-- `datareporting.policy.dataSubmissionEnabled` - allows submission to the
-   server.
-- `datareporting.policy.dataSubmissionPolicyBypassNotification` - bypasses the
-   checks to see if the policy has been shown and agreed to by the user. Set it
-   to `true` for testing.
-- `toolkit.telemetry.log.level` - very handy for watching telemetry activity in
-   the javascript console. `Trace`, `Debug`, `Info`, `Warn`, etc...
+```
+user_pref("toolkit.telemetry.send.overrideOfficialCheck", true);
+```
+
+Without toolkit.telemetry.send.overrideOfficialCheck set, telemetry is only sent for official builds.
+
+### Bypass data policy checks
 
-example (values to paste into prefs.js):
+The data policy checks make sure the user has been shown and
+has accepted the data policy. Bypass them with:
+
+```
+user_pref("datareporting.policy.dataSubmissionPolicyBypassNotification",true);
+user_pref("datareporting.policy.dataSubmissionEnabled", true);
+```
+
+### Enable telemetry tracing
 
 ```
-user_pref("toolkit.telemetry.server", "http://localhost:8080/wibble");
-user_pref("toolkit.telemetry.server_owner", "Nobody");
+user_pref("toolkit.telemetry.log.level", "Trace");
+```
+
+The output will show up on the DevTools console:
+
+    Menu => "Tools" => "Developer Tools" => "Error Console"  (CTRL+SHIFT+J)
+
+If pings aren't showing up, look there for clues.
+
+To log to stdout as well as the console:
+```
+user_pref("toolkit.telemetry.log.dump", true);
+```
+
+### Reduce submission interval
+
+For testing it can be handy to reduce down the submission interval (it's
+usually on the order of hours), eg:
+```
+user_pref("services.sync.telemetry.submissionInterval", 20); // in seconds
+```
+
+### Example user.js file
+
+All the above suggestions in one go, for `$PROFILE/user.js`:
+
+```
+user_pref("toolkit.telemetry.server", "http://localhost:12345");
+user_pref("toolkit.telemetry.server_owner", "TimmyTestfish");
+user_pref("toolkit.telemetry.log.level", "Trace");
+user_pref("toolkit.telemetry.log.dump", true);
+user_pref("toolkit.telemetry.send.overrideOfficialCheck", true);
 user_pref("datareporting.policy.dataSubmissionPolicyBypassNotification",true);
+user_pref("services.sync.telemetry.submissionInterval", 20);
 user_pref("datareporting.policy.dataSubmissionEnabled", true);
-user_pref("toolkit.telemetry.log.level", "Trace");
-user_pref("toolkit.telemetry.send.overrideOfficialCheck", true);
+user_pref("datareporting.healthreport.uploadEnabled",true);
 ```
 
 ## Troubleshooting
 
-### Running a test server
-
-To run a test server locally to dump out the sent data, try
-https://github.com/mozilla/gzipServer
-(or alternatively https://github.com/bcampbell/webhole).
-
-Make sure you set `toolkit.telemetry.server`/`toolkit.telemetry.server_owner`
-to point to your local server.
-
-### Log output
+### Sending test pings
 
-If you've got logging on (eg `user_pref("toolkit.telemetry.log.level", "Trace");`),
-the output will show up on the javascript console:
-
-    Menu => "Tools" => "Developer Tools" => "Error Console"
-
-If data isn't showing up, keep an eye out for messages in the console.
-For example: "Telemetry is not allowed to send pings" is an indication that
-the official-build check is failing (overridden by
-`toolkit.telemetry.send.overrideOfficialCheck`).
-
-### Test pings
-
-From the javascript console, you can force an immediate test ping:
+From the DevTools console, you can send an immediate test ping:
 
 ```
 Cu.import("resource://gre/modules/TelemetrySession.jsm");
-TelemetrySession.testPing()
+TelemetrySession.testPing();
 ```
 
+### Trace message: "Telemetry is not allowed to send pings"
+
+This indicates `TelemetrySend.sendingEnabled()` is returning false;
+
+Fails if not an official build (override using `toolkit.telemetry.send.overrideOfficialCheck`).
+
+If `toolkit.telemetry.unified` and `datareporting.healthreport.uploadEnabled` are true, then
+`sendingEnabled()` returns true;
+
+If `toolkit.telemetry.unified` is false, then the intended-to-be-deprecated `toolkit.telemetry.enabled` controls the result.
+We're using unified telemetry, so this shouldn't be an issue.
+
+### Trace message: "can't send ping now, persisting to disk"
+
+Trace shows:
+```
+TelemetrySend::submitPing - can't send ping now, persisting to disk - canSendNow: false
+```
+
+This means `TelemetryReportingPolicy.canUpload()` is returning false.
+
+Requirements for `canUpload()`:
+
+`datareporting.policy.dataSubmissionEnabled` must be true.
+AND
+`datareporting.policy.dataSubmissionPolicyNotifiedTime` has a sane timestamp (and is > `OLDEST_ALLOWED_ACCEPTANCE_YEAR`).
+AND
+`datareporting.policy.dataSubmissionPolicyAcceptedVersion` >= `datareporting.policy.minimumPolicyVersion`
+
+Or the notification policy can be bypassed by setting:
+`datareporting.policy.dataSubmissionPolicyBypassNotification` to true.
+
 ## Further documentation
 
 The Telemetry documentation index is at:
 
 https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/index.html
 
 There's a good summary of settings (both compile-time and run-time prefs):