Bug 1447057 - Document policies for using third-party Python packages; r=gps
authorDave Hunt <dhunt@mozilla.com>
Mon, 19 Mar 2018 17:53:33 +0000
changeset 412691 53109cc01f60ddbfb7e85d47cc49c3ff92d894de
parent 412690 98d330009c839f8d4d67d84589b1a07a8497265d
child 412692 60de60fe27339fc2dcf5f47a31a68c7e6bdd58b0
push id101981
push useraiakab@mozilla.com
push dateTue, 10 Apr 2018 22:18:59 +0000
treeherdermozilla-inbound@9ad2b8aabfae [default view] [failures only]
perfherder[talos] [build metrics] [platform microbench] (compared to previous push)
reviewersgps
bugs1447057
milestone61.0a1
first release with
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
last release without
nightly linux32
nightly linux64
nightly mac
nightly win32
nightly win64
Bug 1447057 - Document policies for using third-party Python packages; r=gps MozReview-Commit-ID: 8eJt84pEKlv
python/docs/index.rst
python/moz.build
tools/lint/codespell.yml
new file mode 100644
--- /dev/null
+++ b/python/docs/index.rst
@@ -0,0 +1,48 @@
+=================================
+Using third-party Python packages
+=================================
+
+When using third-party Python packages, there are two options:
+
+#. Install/use a vendored version of the package.
+#. Install the package from a package index, such as PyPI or our internal
+   mirror.
+
+Vendoring Python packages
+=========================
+
+If the Python package is to be used in the building of Firefox itself, then we
+**MUST** use a vendored version. This ensures that to build Firefox we only
+require a checkout of the source, and do not depend on a package index. This
+ensures that building Firefox is deterministic and dependable, avoids packages
+from changing out from under us, and means we’re not affected when 3rd party
+services are offline. We don't want a DoS against PyPI or a random package
+maintainer removing an old tarball to delay a Firefox chemspill.
+
+Where possible, the following policy applies to **ALL** vendored packages:
+
+* Vendored libraries **SHOULD NOT** be modified except as required to
+  successfully vendor them.
+* Vendored libraries **SHOULD** be released copies of libraries available on
+  PyPI.
+
+Using a Python package index
+============================
+
+If the Python package is not used in the building of Firefox then it can be
+installed from a package index. Some tasks are not permitted to use external
+resources, and for those we can publish packages to an internal PyPI mirror.
+See `how to upload to internal PyPI <https://wiki.mozilla.org/ReleaseEngineering/How_To/Upload_to_internal_Pypi>`_
+for more details. If you are not restricted, you can install packages from PyPI
+or another package index.
+
+All packages installed from a package index **MUST** specify hashes to ensure
+compatibiliy and protect against remote tampering. Hash-checking mode can be
+forced on when using ``pip`` be specifying the ``--require-hashes``
+command-line option. See `hash-checking mode <https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode>`_ for
+more details.
+
+Note that when using a Python package index there is a risk that the service
+could be unavailable, or packages may be updated or even pulled without notice.
+These issues are less likely with our internal PyPI mirror, but still possible.
+If this is undesirable, then consider vendoring the package.
--- a/python/moz.build
+++ b/python/moz.build
@@ -43,16 +43,18 @@ SPHINX_PYTHON_PACKAGE_DIRS += [
     'mozversioncontrol/mozversioncontrol',
 ]
 
 with Files('**.py'):
     SCHEDULES.inclusive += ['docs']
 
 SPHINX_TREES['mach'] = 'mach/docs'
 
+SPHINX_TREES['python'] = 'docs'
+
 with Files('mach/docs/**'):
     SCHEDULES.exclusive = ['docs']
 
 PYTHON_UNITTEST_MANIFESTS += [
     'mach/mach/test/python.ini',
     'mozbuild/dumbmake/test/python.ini',
     'mozlint/test/python.ini',
     'mozterm/test/python.ini',
--- a/tools/lint/codespell.yml
+++ b/tools/lint/codespell.yml
@@ -1,16 +1,17 @@
 ---
 codespell:
     description: Check code for common misspellings
     include:
         - browser/base/content/docs/
         - browser/experiments/docs
         - build/docs
         - mobile/android/docs
+        - python/docs
         - python/mozlint
         - taskcluster/docs
         - testing/mozbase/docs
         - toolkit/components/extensions/docs
         - toolkit/components/telemetry/docs/
         - toolkit/crashreporter/docs
         - tools/lint
     exclude: