build/checksums.py
author Serban Stanca <sstanca@mozilla.com>
Thu, 17 Jul 2025 20:21:32 +0300 (11 hours ago)
changeset 797003 7ec5a911287f51bc177058928bb102163a3b656e
parent 791331 a56a39ea9e9f7f9d3accb37abab242fd3bfc2eb0
permissions -rwxr-xr-x
Revert "Bug 1977690 - Remove unused AppRequestInterceptor in androidTests r=aaronmt" for causing fenix-debug failures. This reverts commit bc9dc5f4296482e17560627acaacd2797e462211.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
     1
#!/usr/bin/python
94475
Gervase Markham <gerv@gerv.net>
parents: 84558
diff changeset
     2
# This Source Code Form is subject to the terms of the Mozilla Public
Gervase Markham <gerv@gerv.net>
parents: 84558
diff changeset
     3
# License, v. 2.0. If a copy of the MPL was not distributed with this
Gervase Markham <gerv@gerv.net>
parents: 84558
diff changeset
     4
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
     5
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
     6
import hashlib
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
     7
import logging
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
     8
import os
643525
e51d5f374c771cb37f7c63311d0366219dc00e3e Bug 1790816 - Reformat build/ with isort. r=linter-reviewers,ahal DONTBUILD
Marco Castelluccio <mcastelluccio@mozilla.com>
parents: 554551
diff changeset
     9
from optparse import OptionParser
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    10
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    11
logger = logging.getLogger("checksums.py")
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
    12
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    13
414623
1f1186400490312f53369a838d2fdddd594a8ca9 Bug 1455143 - Use a reasonable buffer size for reading files; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414622
diff changeset
    14
def digest_file(filename, digest, chunk_size=131072):
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    15
    """Produce a checksum for the file specified by 'filename'.  'filename'
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    16
    is a string path to a file that is opened and read in this function.  The
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    17
    checksum algorithm is specified by 'digest' and is a valid OpenSSL
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    18
    algorithm.  If the digest used is not valid or Python's hashlib doesn't
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    19
    work, the None object will be returned instead.  The size of blocks
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    20
    that this function will read from the file object it opens based on
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    21
    'filename' can be specified by 'chunk_size', which defaults to 1K"""
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    22
    assert not os.path.isdir(filename), "this function only works with files"
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    23
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    24
    logger.debug("Creating new %s object" % digest)
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    25
    h = hashlib.new(digest)
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    26
    with open(filename, "rb") as f:
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    27
        while True:
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    28
            data = f.read(chunk_size)
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    29
            if not data:
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    30
                logger.debug("Finished reading in file")
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    31
                break
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    32
            h.update(data)
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    33
    hash = h.hexdigest()
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    34
    logger.debug("Hash for %s is %s" % (filename, hash))
414618
Gregory Szorc <gps@mozilla.com>
parents: 402274
diff changeset
    35
    return hash
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    36
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    37
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    38
def process_files(dirs, output_filename, digests):
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    39
    """This function takes a list of directory names, 'drs'. It will then
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    40
    compute the checksum for each of the files in these by by opening the files.
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    41
    Once each file is read and its checksum is computed, this function
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    42
    will write the information to the file specified by 'output_filename'.
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    43
    The path written in the output file will have anything specified by 'strip'
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    44
    removed from the path.  The output file is closed before returning nothing
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
    45
    The algorithm to compute checksums with can be specified by 'digests'
84558
1b89605ede03c96532646cc624fc68a2571eb162 bug 715586: checksums.py should generate sha1 and md5 checksums. r=catlee,ted
Rail Aliiev <rail@mozilla.com>
parents: 62358
diff changeset
    46
    and needs to be a list of valid OpenSSL algorithms.
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    47
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    48
    The output file is written in the format:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    49
        <hash> <algorithm> <filesize> <filepath>
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    50
    Example:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    51
        d1fa09a<snip>e4220 sha1 14250744 firefox-4.0b6pre.en-US.mac64.dmg
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    52
    """
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    53
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    54
    if os.path.exists(output_filename):
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    55
        logger.debug('Overwriting existing checksums file "%s"' % output_filename)
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    56
    else:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    57
        logger.debug('Creating a new checksums file "%s"' % output_filename)
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    58
    with open(output_filename, "w+") as output:
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    59
        for d in dirs:
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    60
            for root, dirs, files in os.walk(d):
791331
a56a39ea9e9f7f9d3accb37abab242fd3bfc2eb0 Bug 1970692 - Pre: Simplify `make upload`; make checksums reproducible. r=firefox-build-system-reviewers,ahochheiden
Nick Alexander <nalexander@mozilla.com>
parents: 647377
diff changeset
    61
                for f in sorted(files):
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    62
                    full = os.path.join(root, f)
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    63
                    rel = os.path.relpath(full, d)
414622
c9cbbb881c78d5d928b3a6453c1f46940c6f3758 Bug 1455143 - Remove code for failing to obtain a hash; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414621
diff changeset
    64
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    65
                    for digest in digests:
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
    66
                        hash = digest_file(full, digest)
414624
1fa5254b9a69c3f96087dbddd116625c74f8a3c3 Bug 1455143 - Use .write() instead of print >>; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414623
diff changeset
    67
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    68
                        output.write(
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    69
                            "%s %s %s %s\n" % (hash, digest, os.path.getsize(full), rel)
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    70
                        )
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    71
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
    72
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    73
def setup_logging(level=logging.DEBUG):
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    74
    """This function sets up the logging module using a speficiable logging
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    75
    module logging level.  The default log level is DEBUG.
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    76
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    77
    The output is in the format:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    78
        <level> - <message>
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    79
    Example:
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    80
        DEBUG - Finished reading in file"""
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    81
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    82
    logger = logging.getLogger("checksums.py")
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    83
    logger.setLevel(logging.DEBUG)
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    84
    handler = logging.StreamHandler()
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    85
    handler.setLevel(level)
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    86
    formatter = logging.Formatter("%(levelname)s - %(message)s")
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    87
    handler.setFormatter(formatter)
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    88
    logger.addHandler(handler)
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    89
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
    90
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    91
def main():
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    92
    """This is a main function that parses arguments, sets up logging
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    93
    and generates a checksum file"""
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    94
    # Parse command line arguments
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
    95
    parser = OptionParser()
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    96
    parser.add_option(
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    97
        "-d",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    98
        "--digest",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
    99
        help="checksum algorithm to use",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   100
        action="append",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   101
        dest="digests",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   102
    )
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   103
    parser.add_option(
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   104
        "-o",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   105
        "--output",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   106
        help="output file to use",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   107
        action="store",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   108
        dest="outfile",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   109
        default="checksums",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   110
    )
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   111
    parser.add_option(
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   112
        "-v",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   113
        "--verbose",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   114
        help="Be noisy (takes precedence over quiet)",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   115
        action="store_true",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   116
        dest="verbose",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   117
        default=False,
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   118
    )
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   119
    parser.add_option(
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   120
        "-q",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   121
        "--quiet",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   122
        help="Be quiet",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   123
        action="store_true",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   124
        dest="quiet",
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   125
        default=False,
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   126
    )
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
   127
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   128
    options, args = parser.parse_args()
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   129
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
   130
    # Figure out which logging level to use
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   131
    if options.verbose:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   132
        loglevel = logging.DEBUG
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   133
    elif options.quiet:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   134
        loglevel = logging.ERROR
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   135
    else:
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   136
        loglevel = logging.INFO
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   137
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
   138
    # Set up logging
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   139
    setup_logging(loglevel)
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   140
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   141
    # Validate the digest type to use
84558
1b89605ede03c96532646cc624fc68a2571eb162 bug 715586: checksums.py should generate sha1 and md5 checksums. r=catlee,ted
Rail Aliiev <rail@mozilla.com>
parents: 62358
diff changeset
   142
    if not options.digests:
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   143
        options.digests = ["sha1"]
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   144
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   145
    for i in args:
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
   146
        if not os.path.isdir(i):
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   147
            logger.error("%s is not a directory" % i)
414625
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
   148
            exit(1)
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
   149
12cfbcd2ccf467c79bb614e3b99cf4a677ff95a6 Bug 1455143 - Refactor checksumming to occur after upload.py; r=ted
Gregory Szorc <gps@mozilla.com>
parents: 414624
diff changeset
   150
    process_files(args, options.outfile, options.digests)
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   151
385817
6074db12d685655fe5692d59471b3c32cc967dc9 Bug 1406650 - Make build/*.py and a few other files flake8 compatible and add them to the list of files to check r=chmanchester
Sylvestre Ledru <sledru@mozilla.com>
parents: 94475
diff changeset
   152
554551
994ae8e4833c90447d91f0e26a718573cff5a514 Bug 1654103: Standardize on Black for Python code in `mozilla-central`.
Ricky Stewart <rstewart@mozilla.com>
parents: 554311
diff changeset
   153
if __name__ == "__main__":
55922
ab6d8c5a300a9f1b54f9d86628d6292ad89d1c15 Bug 578393 - Create a file with checksums for all builds that we upload, r=ted a=gavin
John Ford <jhford@mozilla.com>
parents:
diff changeset
   154
    main()