a49367f0d09b8954493004fa15c64773f0b630a9: overlay: add test for revsets (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Thu, 01 Jun 2017 23:40:06 +0800 - rev 691596
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
overlay: add test for revsets (bug 1365181) r=gps MozReview-Commit-ID: 1nKoeNxS6aa
8e860aa2f36e5bf744aecf543d1d2db56a2e24ee: servo-vcs-sync: add servo-backout-pr services (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Wed, 31 May 2017 15:07:26 +0800 - rev 691595
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: add servo-backout-pr services (bug 1365181) r=gps MozReview-Commit-ID: 9td75hVxXwc
35e9f1dc4c042a32f815bc16d4128707cd98550c: servo-vcs-sync: create servo/servo pull requests from mercurial backouts (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Thu, 04 May 2017 11:36:08 +0800 - rev 691594
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: create servo/servo pull requests from mercurial backouts (bug 1365181) r=gps MozReview-Commit-ID: G58AaYwSDct
e5a6f53e6079df54b51be7da0630c73ff897a56f: servo-vcs-sync: extract hg_converted handling into own method (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Tue, 23 May 2017 23:33:39 +0800 - rev 691593
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: extract hg_converted handling into own method (bug 1365181) r=gps MozReview-Commit-ID: BRVAfG3gabj
4d6b4e5f700bcbf89ade180b0df70d139a9d541e: servo-vcs-sync: move pulse callbacks to global functions (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Tue, 23 May 2017 23:31:47 +0800 - rev 691592
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: move pulse callbacks to global functions (bug 1365181) r=gps MozReview-Commit-ID: FPC7fh1glZD
f52386fce1b32ca9e8a004e648edb18b0fce20d0: servo-vcs-sync: pass config object as extra_data to pulse consumers (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Tue, 23 May 2017 23:24:17 +0800 - rev 691591
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: pass config object as extra_data to pulse consumers (bug 1365181) r=gps MozReview-Commit-ID: Fxfrq109sNV
25eab1192144d77dadb636e7c6b78267a7651b6c: servo-vcs-sync: add extra_data param to pulse consumers (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Tue, 23 May 2017 23:23:10 +0800 - rev 691590
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: add extra_data param to pulse consumers (bug 1365181) r=gps MozReview-Commit-ID: FCHEKPW9Dpu
1554e13984472b25036558012520af0f3fce0bcf: servo-vcs-sync: add configuration for servo-backout-pr (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Wed, 03 May 2017 12:21:33 +0800 - rev 691589
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: add configuration for servo-backout-pr (bug 1365181) r=gps MozReview-Commit-ID: 5BRbrX12sqF
510eccf4bc5b244d80e3f2aa1696870c46514628: servo-vcs-sync: add mozautomation dependancy (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Wed, 03 May 2017 13:27:34 +0800 - rev 691588
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: add mozautomation dependancy (bug 1365181) r=gps MozReview-Commit-ID: Fmh0SJfTAi6
3a111f471c95f52391dfd54eb64f3ab51ccc8656: testing: add hgext to vcssync and hgdev test suite (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Thu, 01 Jun 2017 23:36:02 +0800 - rev 691587
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
testing: add hgext to vcssync and hgdev test suite (bug 1365181) r=gps vcssync and hgdev both use extensions from hgext, and may require changing hgext code. MozReview-Commit-ID: IF0629tn1i9
760cdce02edc272ddd1c0ecabe973c24e6941e19: servo-vcs-sync: create the wheelhouse in a shared location (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Wed, 03 May 2017 13:19:43 +0800 - rev 691586
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: create the wheelhouse in a shared location (bug 1365181) r=gps MozReview-Commit-ID: 1Jt8tuZ3Vqo
69b4ecd96bece2e7c660d80d674534025a19bea4: servo-vcs-sync: fix PEP8 warnings due to long lines (bug 1365181) r=gps
byron jones <glob@mozilla.com> - Thu, 04 May 2017 15:47:34 +0800 - rev 691585
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
servo-vcs-sync: fix PEP8 warnings due to long lines (bug 1365181) r=gps MozReview-Commit-ID: IPCkgnbLxTS
09700eb14c80e42342407534a9287645bfdf26f3: hgtemplates: port gitweb changes for rendering search box
Gregory Szorc <gps@mozilla.com> - Mon, 19 Jun 2017 12:41:14 -0700 - rev 691584
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
hgtemplates: port gitweb changes for rendering search box This is a port of 58f3088aa2f5db6d742a23ec542686fde8f696ac, cb5123eff7d1b50b24495f18dc84bd8d8ea3898c, and 2d93d2159e30e0a19fb23e13d0120e56dda5e8fb from upstream Mercurial to improve the search experience for that theme. Also included are minor changes to pushlog's template to make it compatible with the new layout.
4822814f9f1f4ed4faf54299a68a54937ff75fc0: hghooks: hook to trigger cache population (bug 1358239); r=glob
Gregory Szorc <gps@mozilla.com> - Fri, 09 Jun 2017 15:44:38 -0700 - rev 691583
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
hghooks: hook to trigger cache population (bug 1358239); r=glob Mozilla has historically had problems with the tags cache causing performance problems. Most of these problems were resolved a few years ago by a rewritten tags cache implementation in Mercurial. However, for that cache to work it needs to be populated. And, the cache isn't populated until something attempts to resolve a symbol that could be a tag. This is normally not a problem. On hgweb, the tags cache will be updated on most page views. However, it can be problematic on the master server. Nothing in the regular push flow appears to access tags data. This means that the tags cache may not be written on the master server. It so happens that we have processes running on the master server that don't have write privileges to repos. So while they may trigger tags cache population, the cache I/O fails and the cache is never written. Assuming the cache is never written, over time these processes have to compute more and more tags data and over time the amount of CPU required balloons. Enough time passes and alerts start to fire because processes that should be quick are spending dozens of seconds computing tags data. This commit solves that problem by implementing a pre transaction close hook that accesses tags data, thus ensuring the tags cache is up to date. Other processes should never have to compute tags data again. Furthermore, `hg pull` will transfer tags cache data. So populating the cache before replication ensures that hgweb machines also don't have to populate the cache. Finally, the bundles generated for clone bundles should have current tags caches, ensuring that people who clone from them don't need to regenerate the data. So ensuring the tags cache is populated on the master server is full of wins. The new hook is globally installed on the master server so all repos benefit from tags cache generation. Unless we pre-populate the tags cache on all repos, the first push to a repo after this is enabled may be slow. All subsequent pushes may slow down because of this hook. The tags cache population time is proportional to the number of files in a repo and the number of heads being pushed. However, the common case is 1 head per push and Mercurial should cache the manifest for that head, thus assuring rapid tags cache generation. The tags cache generation time is recorded in blackbox logs. So if the logs show this hook causes too much of a perf drain, we can look into alternate mechanisms for populating the tags cache. MozReview-Commit-ID: 5q9TuqO4JaS
3986d9ddbc0886ed5837de845fec95e0ca48dfdb: pushlog: optimize revision resolution (bug 1337946); r=glob
Gregory Szorc <gps@mozilla.com> - Mon, 19 Jun 2017 11:41:56 -0700 - rev 691582
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
pushlog: optimize revision resolution (bug 1337946); r=glob Some revsets iterate over the pushlog and emit revisions. Since the pushlog API returns nodes, they need to convert nodes to integer revs. Previously, they were using repo[x].rev() to resolve a node to a rev. This constructs a full changectx type, which can be expensive. We rewrite the code to perform node -> rev translations in the most efficient manner possible. The impact on revset performance for mozilla-central is clearly visible: $ hg perfrevset 'pushhead()' ! wall 1.108759 comb 1.110000 user 1.100000 sys 0.010000 (best of 9) ! wall 0.752788 comb 0.750000 user 0.740000 sys 0.010000 (best of 14 $ hg perfrevset 'pushdate(">2017")' ! wall 1.026203 comb 1.020000 user 1.020000 sys 0.000000 (best of 10) ! wall 0.722270 comb 0.720000 user 0.710000 sys 0.010000 (best of 14) $ hg perfrevset 'pushuser(gszorc)' ! wall 0.788897 comb 0.790000 user 0.790000 sys 0.000000 (best of 13) ! wall 0.716771 comb 0.720000 user 0.700000 sys 0.020000 (best of 14) MozReview-Commit-ID: 6Od0xOVBcav
5bc8e4314613ecd8f6398c566d736399c9d58fda: pushlog: expose pushlog revsets to hgweb (bug 1337946); r=glob
Gregory Szorc <gps@mozilla.com> - Thu, 08 Jun 2017 18:39:25 -0700 - rev 691581
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
pushlog: expose pushlog revsets to hgweb (bug 1337946); r=glob Revsets must explicitly opt in to being exposed on hgweb for performance/security reasons. When we implemented the pushlog revsets, I'm pretty sure I just forgot to enable (at least some of) them on hgweb. Some of the pushlog revsets are a bit expensive to compute. pushhead(), pushdate(), and pushuser() each query the full database then iterate on results to do filtering. For a repo like mozilla-central which has ~32,000 pushes, `hg perfrevset` says this takes ~1.11s on my i7-6700k. Other built-in and enabled revsets are already more expensive. So I think this is tolerable. MozReview-Commit-ID: FKLEk4OlQnB
c23d48384dbaf1604c6cbe59290b6fe510067622: hgmo: don't send list of push nodes with automationrelevant response (bug 1303904); r=glob
Gregory Szorc <gps@mozilla.com> - Thu, 08 Jun 2017 13:33:33 -0700 - rev 691580
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
hgmo: don't send list of push nodes with automationrelevant response (bug 1303904); r=glob This data is redundant with other data in the JSON. Furthermore, for large pushes (such as uplifts, which contains thousands of changesets), the redundancy is massive. I'm pretty sure nobody looks at this data. The only consumer of this API I'm aware of (TaskCluster decision task) only cares about the "node," "desc," and "files" elements. For json-automationrelevance/5dddbefdf759f09b1411f33fa0920835b919fc81 against the mozilla-aurora repo, this change reduces the raw HTTP response size from 2348 MB to 10.8 MB and the server-side CPU required from ~311s to ~6s (as measured on my i7-6700K). Yes, you read that correctly. MozReview-Commit-ID: 4Q1VfAllOis
af4420a6680e35cdb8ba00ec404be9abfd2bc7b1: pushlog: API to cache pushlog lookups (bug 1303904); r=glob
Gregory Szorc <gps@mozilla.com> - Thu, 08 Jun 2017 13:32:37 -0700 - rev 691579
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
pushlog: API to cache pushlog lookups (bug 1303904); r=glob The pushlog API is not conducive to performance because various methods result in a SQLite database handle being opened for every lookup. When doing things like looking up pushlog entries for every node in a list, the overhead can really add up. This commit introduces a context manager for populating a cache to facilitate rapid node -> push lookups. This commit is somewhat hacky. There are a number of alternatives I'd prefer, such as a bulk lookup API or automatic caching. A bulk lookup API isn't usable in the case of the automationrelevance web command because the function that calls the pushlog API is per changeset, not per collection. And I'm not yet willing to implement automatic caching because caching is hard and I don't want to invite bugs. A context manager, however, allows us to create caches with scoped lifetimes thus limiting the caching (and bugs related to it) to consumers who opt into the cache. Yes, it's hacky. But it gets the job done. To prove the cache works, the automationrelevance web command has been updated to use it. This drops time to first byte on json-automationrelevance/5dddbefdf759f09b1411f33fa0920835b919fc81 HTTP requests against the mozilla-aurora repo from ~80s to ~1.5s. However, there is still a major source of clownshoes for that request... MozReview-Commit-ID: 37bP49sHxKZ
cbe2a5a5ae20e927791f5ad6a4e99d76f012c55b: hgmo: refactor loop iterating over revs (bug 1303904); r=glob
Gregory Szorc <gps@mozilla.com> - Thu, 08 Jun 2017 13:31:53 -0700 - rev 691578
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
hgmo: refactor loop iterating over revs (bug 1303904); r=glob We'll soon introduce a second consumer of this result. So we save the revset result to a variable to avoid redundant execution. We also buffer the result in a list because we know we'll be iterating twice and this will be more efficient than iterating a revset result type. MozReview-Commit-ID: IjAK6qMV8eA
01497dd90af6e26f91295ffb763ae811091dc00f: pushlog: factor out code to obtain a push from a node (bug 1303904); r=glob
Gregory Szorc <gps@mozilla.com> - Thu, 08 Jun 2017 11:24:18 -0700 - rev 691577
Push 87314 by gszorc@mozilla.com at Thu, 02 Nov 2017 02:08:10 +0000
pushlog: factor out code to obtain a push from a node (bug 1303904); r=glob A subsequent commit will introduce a consumer that wants to obtain push records for multiple nodes. To facilitate doing this with a single database handle, refactor the code into a function that takes a database connection as an argument. MozReview-Commit-ID: 3P6tLIz4C4N
(0) -300000 -100000 -30000 -10000 -3000 -1000 -300 -100 -50 -20 +20 +50 +100 +300 +1000 +3000 +10000 +30000 +100000 tip