streamclone: consider secret changesets (BC) (issue5589)
authorGregory Szorc <gregory.szorc@gmail.com>
Fri, 09 Jun 2017 10:41:13 -0700
changeset 38000 33b7283a38284e65e9ec78dc73566ba919267186
parent 37999 f924dd04397460a95ecfbea8a2f2b1a3af97e088
child 38001 23734c0e361fa19c1c580672f0f06d9117e367e6
push id528
push usergszorc@mozilla.com
push dateThu, 15 Jun 2017 18:20:03 +0000
streamclone: consider secret changesets (BC) (issue5589) Previously, a repo containing secret changesets would be served via stream clone, transferring those secret changesets. While secret changesets aren't meant to imply strong security (if you really want to keep them secret, others shouldn't have read access to the repo), we should at least make an effort to protect secret changesets when possible. After this commit, we no longer serve stream clones for repos containing secret changesets by default. This is backwards incompatible behavior. In case anyone is relying on the behavior, we provide a config option to opt into the old behavior. Note that this defense is only beneficial for remote repos accessed via the wire protocol: if a client has access to the files backing a repo, they can get to the raw data and see secret revisions.
mercurial/help/config.txt
mercurial/streamclone.py
mercurial/wireproto.py
tests/test-clone-uncompressed.t
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -1653,16 +1653,20 @@ Controls generic server settings.
     server and client. Over a LAN (100 Mbps or better) or a very fast
     WAN, an uncompressed streaming clone is a lot faster (~10x) than a
     regular clone. Over most WAN connections (anything slower than
     about 6 Mbps), uncompressed streaming is slower, because of the
     extra data transfer overhead. This mode will also temporarily hold
     the write lock while determining what data to transfer.
     (default: True)
 
+``uncompressedallowsecret``
+    Whether to allow stream clones when the repository contains secret
+    changesets. (default: False)
+
 ``preferuncompressed``
     When set, clients will try to use the uncompressed streaming
     protocol. (default: False)
 
 ``disablefullbundle``
     When set, servers will refuse attempts to do pull-based clones.
     If this option is set, ``preferuncompressed`` and/or clone bundles
     are highly recommended. Partial clones will still be allowed.
--- a/mercurial/streamclone.py
+++ b/mercurial/streamclone.py
@@ -8,16 +8,17 @@
 from __future__ import absolute_import
 
 import struct
 
 from .i18n import _
 from . import (
     branchmap,
     error,
+    phases,
     store,
     util,
 )
 
 def canperformstreamclone(pullop, bailifbundle2supported=False):
     """Whether it is possible to perform a streaming clone as part of pull.
 
     ``bailifbundle2supported`` will cause the function to return False if
@@ -157,19 +158,28 @@ def maybeperformlegacystreamclone(pullop
         repo._applyopenerreqs()
         repo._writerequirements()
 
         if rbranchmap:
             branchmap.replacecache(repo, rbranchmap)
 
         repo.invalidate()
 
-def allowservergeneration(ui):
+def allowservergeneration(repo):
     """Whether streaming clones are allowed from the server."""
-    return ui.configbool('server', 'uncompressed', True, untrusted=True)
+    if not repo.ui.configbool('server', 'uncompressed', True, untrusted=True):
+        return False
+
+    # The way stream clone works makes it impossible to hide secret changesets.
+    # So don't allow this by default.
+    secret = phases.hassecret(repo)
+    if secret:
+        return repo.ui.configbool('server', 'uncompressedallowsecret', False)
+
+    return True
 
 # This is it's own function so extensions can override it.
 def _walkstreamfiles(repo):
     return repo.store.walk()
 
 def generatev1(repo):
     """Emit content for version 1 of a streaming clone.
 
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -749,17 +749,17 @@ def _capabilities(repo, proto):
     computation
 
     - returns a lists: easy to alter
     - change done here will be propagated to both `capabilities` and `hello`
       command without any other action needed.
     """
     # copy to prevent modification of the global list
     caps = list(wireprotocaps)
-    if streamclone.allowservergeneration(repo.ui):
+    if streamclone.allowservergeneration(repo):
         if repo.ui.configbool('server', 'preferuncompressed', False):
             caps.append('stream-preferred')
         requiredformats = repo.requirements & repo.supportedformats
         # if our local revlogs are just revlogv1, add 'stream' cap
         if not requiredformats - {'revlogv1'}:
             caps.append('stream')
         # otherwise, add 'streamreqs' detailing our local revlog format
         else:
@@ -941,17 +941,17 @@ def pushkey(repo, proto, namespace, key,
     return '%s\n' % int(r)
 
 @wireprotocommand('stream_out')
 def stream(repo, proto):
     '''If the server supports streaming clone, it advertises the "stream"
     capability with a value representing the version and flags of the repo
     it is serving. Client checks to see if it understands the format.
     '''
-    if not streamclone.allowservergeneration(repo.ui):
+    if not streamclone.allowservergeneration(repo):
         return '1\n'
 
     def getstream(it):
         yield '0\n'
         for chunk in it:
             yield chunk
 
     try:
--- a/tests/test-clone-uncompressed.t
+++ b/tests/test-clone-uncompressed.t
@@ -44,16 +44,87 @@ Clone with background file closing enabl
   sending getbundle command
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
   bundle2-input-part: total payload size 58
   bundle2-input-part: "listkeys" (params: 1 mandatory) supported
   bundle2-input-bundle: 1 parts total
   checking for updated bookmarks
 
+Cannot stream clone when there are secret changesets
+
+  $ hg -R server phase --force --secret -r tip
+  $ hg clone --uncompressed -U http://localhost:$HGPORT secret-denied
+  warning: stream clone requested but server has them disabled
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+
+  $ killdaemons.py
+
+Streaming of secrets can be overridden by server config
+
+  $ cd server
+  $ hg --config server.uncompressedallowsecret=true serve -p $HGPORT -d --pid-file=hg.pid
+  $ cat hg.pid > $DAEMON_PIDS
+  $ cd ..
+
+  $ hg clone --uncompressed -U http://localhost:$HGPORT secret-allowed
+  streaming all changes
+  1027 files to transfer, 96.3 KB of data
+  transferred 96.3 KB in * seconds (*/sec) (glob)
+  searching for changes
+  no changes found
+
+  $ killdaemons.py
+
+Verify interaction between preferuncompressed and secret presence
+
+  $ cd server
+  $ hg --config server.preferuncompressed=true serve -p $HGPORT -d --pid-file=hg.pid
+  $ cat hg.pid > $DAEMON_PIDS
+  $ cd ..
+
+  $ hg clone -U http://localhost:$HGPORT preferuncompressed-secret
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+
+  $ killdaemons.py
+
+Clone not allowed when full bundles disabled and can't serve secrets
+
+  $ cd server
+  $ hg --config server.disablefullbundle=true serve -p $HGPORT -d --pid-file=hg.pid
+  $ cat hg.pid > $DAEMON_PIDS
+  $ cd ..
+
+  $ hg clone --uncompressed http://localhost:$HGPORT secret-full-disabled
+  warning: stream clone requested but server has them disabled
+  requesting all changes
+  remote: abort: server has pull-based clones disabled
+  abort: pull failed on remote
+  (remove --pull if specified or upgrade Mercurial)
+  [255]
+
+Local stream clone with secrets involved
+(This is just a test over behavior: if you have access to the repo's files,
+there is no security so it isn't important to prevent a clone here.)
+
+  $ hg clone -U --uncompressed server local-secret
+  warning: stream clone requested but server has them disabled
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
 
 Stream clone while repo is changing:
 
   $ mkdir changing
   $ cd changing
 
 extension for delaying the server process so we reliably can modify the repo
 while cloning