hgmo: optimize Docker image generation We use the new feature to limit which files are added to the context. We also consolidate the number of instructions in the Dockerfiles to make image building faster. The images still don't build as quickly as I would like. We may have to move instructions into a shell script that is executed inside the container.

docker: support adding a subset of files to the tar archive Profiling revealed that adding all files from v-c-t to the tar archive used for the build context slowed down image generation. This patch adds support for specifying a subset of files to include in the archive.

testing: suppress stderr from hg invocation This was resulting in a warning about obsolescence markers being present since HGRCPATH was set to /dev/null and evolution wasn't enabled.

hgmo: add create-repo script This should have been added in a previous commit.

pash: add a test verifying that `hg serve` is handled Currently only testing clone/pull only. Tests for push will be added later.

hgmo: ability to create repositories on the cluster This is extremely simple. I haven't verified replication is working yet. But it unblocks the next round of tests.

hgmaster: add missing extension files to Docker image pushlog-legacy and serverlog were failing to load due to missing files. Correct that.

hgmo: fix creating SSH keys when creating LDAP users The LDAP package is very sensitive about types. We must pass str or it barfs. Before this patch, we were passing unicode.

hgmaster: add vct files towards the end of the Dockerfile For performance reasons. These files likely change more than others, invalidating the cache.

pash: add basic tests for pash using hgmo Docker cluster Now that we have a Docker cluster for hg.mozilla.org, let's put it to use by testing pash. The implemented test covers basic SSH auth behavior. We have to start somewhere.

hgmo: forcefully shut down containers during clean() Since clean() will remove the containers, it doesn't matter if they shut down gracefully or not. Force the shutdown and enable a quicker shutdown. This will make tests faster once we start testing hgmo clusters.

hgmo: grab Docker images from environment Similar to how mozreview works, we want the images generated by run-mercurial-tests.py to be automatically used in tests without having to incur semi-expensive and redundant image generation.

testing: build all Docker images when running tests

docker: add an API to build all images for all projects

hgmo: ability to specify which port the master SSH server listens on

testing: vastly improve support for running hg.mozilla.org clusters Support for running an hg.mozilla.org has been vastly improved. An "hgmo" mach driver for controlling hg.mozilla.org instances has been created. There is a new API behind it for controlling instances. The hgmaster Docker environment has been greatly improved. It now integrates with an OpenLDAP server. OpenSSH LPK is installed and SSH logins are processed through LDAP lookups. It "just works." SSH requests into the container result in pash successfully being invoked. Support for running hgweb containers has been added. It isn't super well-tested right now. But the master container can SSH into hgweb containers as the hg user and get the desired mirror-pull behavior. Although, things have not been tested to verify mirror-pull and other various SSH connectivity works as expected. Things are probably significantly broken.

pash: replace doc_root with DOC_ROOT; r=bkero We no longer need a dictionary to hold document root instances because we only support 1 server (since we dropped hg.ecmascript.org). This patch replaces doc_root with DOC_ROOT.

pash: remove hg.ecmascript.org support; r=bkero hg.ecmascript.org is no more. Remove the code referencing it.

pash: move non-root login code path to isolated function (bug 1140214); r=bkero We want the root login code path to be clean so that things like import errors don't result in root login being denied. This commit moves the non-root login code path to a separate function. We also move imports into this function so they aren't evaluated until a non-root login is performed. With this commit, the root login code path is now only requires a pash.py file with proper Python syntax and a working Python installation (since all we import are modules from the core Python distribution).

pash: move QuoteForPOSIX into pash.py; r=bkero QuoteForPOSIX is only used by pash.py. Since the system can get borked if pash.py ever gets borked, dropping the import dependency on sh_helper.py helps mitigate our risk that bad code in sh_helper.py will result in a locked out system.