79fa6ea6a9a7981a0576763e427f525e04b18936: testing: securely install Python packages
Gregory Szorc <gps@mozilla.com> - Mon, 29 Feb 2016 14:27:09 -0800 - rev 362337
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
testing: securely install Python packages Before, we relied on the virtualenv on the system. We also upgraded pip and setuptools via normal pip foo, implicitly trusting the server. We also didn't pin hashes in test-requirements.txt. This commit changes all that. We now download virtualenv, setuptools, and pip from our S3 package server. The hashes are pinned so we can detect tampering. Hashes have also been added to test-requirements.txt. Some packages have multiple hashes because OS X and Linux will download different archives. lxml has been upgraded to 3.5.0 due to issues building the existing version on Ubuntu.
7b9b58e346dff827a146de78edf4a626725d5989: git-mozreview: treat commit messages as byte sequences (bug 1251602); r=dminor
Gregory Szorc <gps@mozilla.com> - Mon, 29 Feb 2016 11:47:52 -0800 - rev 362336
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
git-mozreview: treat commit messages as byte sequences (bug 1251602); r=dminor More Unicode foo. We have unicode_literals enabled in git-mozreview. We were opening the commit message file in binary mode but when we performed some string operations with '' literals, Python would try to convert the unicode literal to str using the ascii encoding and the world would blow up. We use b'' literals to prevent implicit type conversion. A test demonstrating bytes preservation has been added. MozReview-Commit-ID: CTL5sXUkd1W
f07e02773a0a03a4bfbd2bf41907ea55809f7dc7: ansible/mozreview: Install npm in virtualenv.
Mark Cote <mcote@mozilla.com> - Mon, 29 Feb 2016 12:15:05 -0500 - rev 362335
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible/mozreview: Install npm in virtualenv.
2d5d156dcb24ac07619745115b80a77c64b68a26: mozreview: draw attention to review comments (bug 1101611) r=mdoglio
byron jones <glob@mozilla.com> - Tue, 23 Feb 2016 15:34:50 +0800 - rev 362334
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
mozreview: draw attention to review comments (bug 1101611) r=mdoglio Review comments appear only as a tiny number in the left margin. This makes them easy to overlook, especially when viewing side-by-side diffs where your focus is on the right column. This adds a dotted line across the diff view to draw attention to the comments. MozReview-Commit-ID: EohHRRTY9pB
e1c7901963e78bba147e1dcf1338c1082a049ae2: mozreview: fix indentation (bug 1101611) r=mdoglio
byron jones <glob@mozilla.com> - Tue, 23 Feb 2016 13:31:07 +0800 - rev 362333
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
mozreview: fix indentation (bug 1101611) r=mdoglio MozReview-Commit-ID: 3V2VdzlT0AZ
ae18684189a0f3c6045f5a8aa08ee579a8d715c2: docker/bmoweb: use socket for connecting to MySQL (bug 1246634)
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 14:09:38 -0800 - rev 362332
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
docker/bmoweb: use socket for connecting to MySQL (bug 1246634) We run MySQL inside the container. We don't need to involve TCP sockets. Let's connect to the server over the UNIX socket.
aca609c335630243fbac9d5540bb9aa99544e3de: docker/bmoweb: explicitly wait on mysqld process to exit (bug 1246634)
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 13:58:42 -0800 - rev 362331
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
docker/bmoweb: explicitly wait on mysqld process to exit (bug 1246634) I /think/ there is a race condition here since terminate() only sends a signal and doesn't wait on the process to actually exit.
52aaee1436b7d4cf07bce337287600135396ea9c: docker/bmoweb: start mysql after cpan module installation (bug 1246634)
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 13:58:01 -0800 - rev 362330
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
docker/bmoweb: start mysql after cpan module installation (bug 1246634) For reasons I can't explain, failure in cpanm will result in the mysqld process terminating. I suspect a signal is being sent to mysqld somehow. Perhaps subprocess.check_call doesn't trap signal handlers like I thought it did...
8e7b9b7313633d975aff16ccd2cd9e92c0d87627: docker/bmoweb: use mysqld instead of mysqld_safe (bug 1246634)
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 13:40:32 -0800 - rev 362329
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
docker/bmoweb: use mysqld instead of mysqld_safe (bug 1246634)
28a39ce8a3b75a654e77ad62a0c2a40d55d249a9: ansible/mozreview: install Python 2.7
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 11:35:08 -0800 - rev 362328
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible/mozreview: install Python 2.7 The MozReview web heads are still running Python 2.6 from the system Python installation. We want to run Python 2.7 everywhere. The first step towards transitioning rbweb to Python 2.7 is to get Python 2.7 installed. We do this by establishing a new "rbweb" role and have it install Python 2.7. We add this role in various locations. It's worth noting that the admin and web machines are currently more under the control of Puppet than Ansible. We'll need to slowly bring more things under Ansible control in order to switch to actually running things from Python 2.7.
4e7f7319a9284e5ad51be0e13222470f911ed862: ansible/mozreview: install mod_wsgi in virtualenv
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 11:34:57 -0800 - rev 362327
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible/mozreview: install mod_wsgi in virtualenv As part of the transition to Python 2.7, we need to build mod_wsgi from source because we don't have an appropriate RPM for it available to us. We already do this for the hg servers. So there is nothing conceptually new here.
8ff9a81ce4622e01d469fe9772897e8261809859: ansible/mozreview: virtualenv fixups to support pip 8
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 11:10:12 -0800 - rev 362326
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible/mozreview: virtualenv fixups to support pip 8 We weren't upgrading the pip in already created virtualenvs. Ensure we do that. This patch effectively copied a lot of code from virtualenv.yml. We can likely kill this once we're running Python 2.7.
3be26461defa1727926b96982208bcf08ea506c1: hgweb: update system packages in chroot Dockerfile
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 10:52:18 -0800 - rev 362325
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
hgweb: update system packages in chroot Dockerfile We want to run the latest packages so we pull in security fixes.
e0f96efa0883b7743a14e7cab411e3451665eb70: hgweb: fix typo in setuptools path in chroot Dockerfile
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 10:51:53 -0800 - rev 362324
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
hgweb: fix typo in setuptools path in chroot Dockerfile
0305a960aa02ff4aacf2bad075c2e58968721efb: ansible/hg: use pip 8 (bug 1243501); r=fubar
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 10:21:12 -0800 - rev 362323
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible/hg: use pip 8 (bug 1243501); r=fubar pip 8 contains peep's hash verification functionality. As part of this commit, we introduce a standalone task for configuring a virtualenv. We replace the virtualenv muckery with it. And we update the requirements.txt files to pip 8's syntax. After this patch, only Autoland is using peep. The reason it wasn't converted is because it is using its own mechanism and wasn't easily ported like everything else. It can be done later. There is a non-zero chance that this will break a prod deploy of MozReview and/or hg.mo. We should be cautious the first time we deploy. fubar reviewed the first version of this commit. It had to be reworked a bit to incorporate other changes that were made since, such as the addition of Pygments to some virtualenvs. MozReview-Commit-ID: F70nbqBd4xK
c0389190eb41403820eea5c7595946cf66dd1508: ansible: install Ansible securely in CentOS Docker image
Gregory Szorc <gps@mozilla.com> - Mon, 22 Feb 2016 23:31:08 -0800 - rev 362322
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible: install Ansible securely in CentOS Docker image By using pip 8's hash pinning functionality. MozReview-Commit-ID: KeKo6gGFuE3
20f7674ee8b71faaf917579e5e03c110538fa1c4: ansible: securely install pip 8.0.3 and setuptools 20.1.1 in CentOS Docker image
Gregory Szorc <gps@mozilla.com> - Fri, 26 Feb 2016 10:04:21 -0800 - rev 362321
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible: securely install pip 8.0.3 and setuptools 20.1.1 in CentOS Docker image Previously we were running older versions of both. We're trying to upgrade to pip 8 everywhere since it has hash verification built-in. As part of switching to pip 8, we add a verifying URL downloader to protect against tampering. We also switch URLs to our S3 bucket, which is under our control and should be more reliable. MozReview-Commit-ID: 3HfiLCF9qWN
5d4fcdd28be544fac0bab646f9ba8612f09cdd63: Disabling Autoland through hostingservice admin page does not work (bug 1250263) r=mdoglio
Dan Minor <dminor@mozilla.com> - Wed, 24 Feb 2016 10:09:16 -0800 - rev 362320
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
Disabling Autoland through hostingservice admin page does not work (bug 1250263) r=mdoglio Apparently for these to pass validation with a 'False' value we have to make them not required. MozReview-Commit-ID: 6L73sfF6KVR
444f8ab38221a647e26aa9701f13569c32d461d7: ansible: upgrade to peep 3.1.1
Gregory Szorc <gps@mozilla.com> - Mon, 22 Feb 2016 22:03:22 -0800 - rev 362319
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
ansible: upgrade to peep 3.1.1 This will allow us to upgrade the ancient pip version installed in CentOS images, which will allow us to stop using peep since its features are baked into pip 8. MozReview-Commit-ID: FOqb4XdSYyF
7d3c184e2ea2bd5cf7228e8912be4c54dbef8a92: reviewboard: disable demand importer when importing pushhooks module
Gregory Szorc <gps@mozilla.com> - Tue, 23 Feb 2016 12:09:41 -0800 - rev 362318
Push 16998 by rwood@mozilla.com at Mon, 02 May 2016 19:42:03 +0000
reviewboard: disable demand importer when importing pushhooks module Without this, upgrading to latest setuptools and pip somehow triggers an "cannot import _imp" error deeps inside pkg_resources code as part of processing rbtools imports. It's a really wonky error. I may submit a patch upstream to have Mercurial's demand importer ignore _imp. MozReview-Commit-ID: LPGbRj471N5
(0) -300000 -100000 -30000 -10000 -3000 -1000 -300 -100 -50 -20 +20 +50 +100 +300 +1000 +3000 +10000 +30000 +100000 +300000 tip