* cookify project

* update travis

* fix up manifest

* fix import sorting

* disable pypy, py36 on travis for now

* fix up readme
This commit is contained in:
Chris AtLee 2016-12-23 11:06:56 -05:00 коммит произвёл GitHub
Родитель 958aad5fb3
Коммит 88fc0c6010
45 изменённых файлов: 1167 добавлений и 102 удалений

Просмотреть файл

@ -1,22 +1,40 @@
language: python
python: '3.5'
sudo: false
env:
global:
- LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so
- SEGFAULT_SIGNALS=all
matrix:
- TOXENV=check
- TOXENV=docs
matrix:
include:
- python: "2.7"
env: TOXENV=flake8
- python: "2.7"
env: TOXENV=py27
- python: "3.4"
env: TOXENV=py34
- python: "3.5"
env: TOXENV=py35
- TOXENV=py27,coveralls,codecov
- TOXENV=py34,coveralls,codecov
- TOXENV=py35,coveralls,codecov
#- TOXENV=py36,coveralls,codecov
#- TOXENV=pypy,coveralls,codecov
before_install:
- python --version
- uname -a
- lsb_release -a
install:
- travis_retry pip install tox coveralls
script: tox
after_success:
- coveralls debug
- coveralls
- pip install tox
- virtualenv --version
- easy_install --version
- pip --version
- tox --version
script:
- tox -v
after_failure:
- more .tox/log/* | cat
- more .tox/*/log/* | cat
before_cache:
- rm -rf $HOME/.cache/pip/log
cache:
directories:
- $HOME/.cache/pip
notifications:
email:
on_success: never
on_failure: always

5
AUTHORS.rst Normal file
Просмотреть файл

@ -0,0 +1,5 @@
Authors
=======
* Chris AtLee - https://atlee.ca

8
CHANGELOG.rst Normal file
Просмотреть файл

@ -0,0 +1,8 @@
Changelog
=========
2.0 (2016-12-16)
-----------------------------------------
* First release on PyPI.

90
CONTRIBUTING.rst Normal file
Просмотреть файл

@ -0,0 +1,90 @@
============
Contributing
============
Contributions are welcome, and they are greatly appreciated! Every
little bit helps, and credit will always be given.
Bug reports
===========
When `reporting a bug <https://github.com/mozilla/build-mar/issues>`_ please include:
* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.
Documentation improvements
==========================
mar could always use more documentation, whether as part of the
official mar docs, in docstrings, or even on the web in blog posts,
articles, and such.
Feature requests and feedback
=============================
The best way to send feedback is to file an issue at https://github.com/mozilla/build-mar/issues.
If you are proposing a feature:
* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that code contributions are welcome :)
Development
===========
To set up `build-mar` for local development:
1. Fork `build-mar <https://github.com/mozilla/build-mar>`_
(look for the "Fork" button).
2. Clone your fork locally::
git clone git@github.com:your_name_here/build-mar.git
3. Create a branch for local development::
git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
4. When you're done making changes, run all the checks, doc builder and spell checker with `tox <http://tox.readthedocs.org/en/latest/install.html>`_ one command::
tox
5. Commit your changes and push your branch to GitHub::
git add .
git commit -m "Your detailed description of your changes."
git push origin name-of-your-bugfix-or-feature
6. Submit a pull request through the GitHub website.
Pull Request Guidelines
-----------------------
If you need some code review or feedback while you're developing the code just make the pull request.
For merging, you should:
1. Include passing tests (run ``tox``) [1]_.
2. Update documentation when there's new API, functionality etc.
3. Add a note to ``CHANGELOG.rst`` about the changes.
4. Add yourself to ``AUTHORS.rst``.
.. [1] If you don't have all the necessary python versions available locally you can rely on Travis - it will
`run the tests <https://travis-ci.org/mozilla/build-mar/pull_requests>`_ for each change you add in the pull request.
It will be slower though ...
Tips
----
To run a subset of tests::
tox -e envname -- py.test -k test_myfeature
To run all the test environments in *parallel* (you need to ``pip install detox``)::
detox

Просмотреть файл

Просмотреть файл

@ -1,3 +1,27 @@
include README.md
graft docs
graft examples
graft src
graft ci
graft tests
include .bumpversion.cfg
include .coveragerc
include .cookiecutterrc
include .editorconfig
include .isort.cfg
include AUTHORS.rst
include CHANGELOG.rst
include CONTRIBUTING.rst
include LICENSE
include README.rst
include get_mozilla_keys.sh
include requirements.txt
prune dist
include requirements.in
include test-requirements.txt
include test-requirements.in
include tox.ini .travis.yml appveyor.yml
global-exclude *.py[cod] __pycache__ *.so *.dylib

Просмотреть файл

@ -1,4 +0,0 @@
Utility for managing mar files. MAR file format is documented at https://wiki.mozilla.org/Software_Update:MAR
[![Coverage Status](https://coveralls.io/repos/mozilla/build-mar/badge.svg?branch=master&service=github)](https://coveralls.io/github/mozilla/build-mar?branch=master)
[![Build Status](https://travis-ci.org/mozilla/build-mar.svg)](https://travis-ci.org/mozilla/build-mar)

103
README.rst Normal file
Просмотреть файл

@ -0,0 +1,103 @@
========
Overview
========
.. start-badges
.. list-table::
:stub-columns: 1
* - docs
- |docs|
* - tests
- | |travis| |appveyor| |requires|
| |coveralls| |codecov|
* - package
- |version| |downloads| |wheel| |supported-versions| |supported-implementations|
.. |docs| image:: https://readthedocs.org/projects/mar/badge/?style=flat
:target: https://readthedocs.org/projects/mar
:alt: Documentation Status
.. |travis| image:: https://travis-ci.org/mozilla/build-mar.svg?branch=master
:alt: Travis-CI Build Status
:target: https://travis-ci.org/mozilla/build-mar
.. |appveyor| image:: https://ci.appveyor.com/api/projects/status/github/mozilla/build-mar?branch=master&svg=true
:alt: AppVeyor Build Status
:target: https://ci.appveyor.com/project/mozilla/build-mar
.. |requires| image:: https://requires.io/github/mozilla/build-mar/requirements.svg?branch=master
:alt: Requirements Status
:target: https://requires.io/github/mozilla/build-mar/requirements/?branch=master
.. |coveralls| image:: https://coveralls.io/repos/mozilla/build-mar/badge.svg?branch=master&service=github
:alt: Coverage Status
:target: https://coveralls.io/r/mozilla/build-mar
.. |codecov| image:: https://codecov.io/github/mozilla/build-mar/coverage.svg?branch=master
:alt: Coverage Status
:target: https://codecov.io/github/mozilla/build-mar
.. |version| image:: https://img.shields.io/pypi/v/mar.svg?style=flat
:alt: PyPI Package latest release
:target: https://pypi.python.org/pypi/mar
.. |downloads| image:: https://img.shields.io/pypi/dm/mar.svg?style=flat
:alt: PyPI Package monthly downloads
:target: https://pypi.python.org/pypi/mar
.. |wheel| image:: https://img.shields.io/pypi/wheel/mar.svg?style=flat
:alt: PyPI Wheel
:target: https://pypi.python.org/pypi/mar
.. |supported-versions| image:: https://img.shields.io/pypi/pyversions/mar.svg?style=flat
:alt: Supported versions
:target: https://pypi.python.org/pypi/mar
.. |supported-implementations| image:: https://img.shields.io/pypi/implementation/mar.svg?style=flat
:alt: Supported implementations
:target: https://pypi.python.org/pypi/mar
.. end-badges
Package for handling Mozilla Archive files. MAR file format is documented at https://wiki.mozilla.org/Software_Update:MAR
* Free software: MPL 2.0 license
Installation
============
::
pip install mar
Documentation
=============
https://mar.readthedocs.io/
Development
===========
To run the all tests run::
tox
Note, to combine the coverage data from all the tox environments run:
.. list-table::
:widths: 10 90
:stub-columns: 1
- - Windows
- ::
set PYTEST_ADDOPTS=--cov-append
tox
- - Other
- ::
PYTEST_ADDOPTS=--cov-append tox

72
appveyor.yml Normal file
Просмотреть файл

@ -0,0 +1,72 @@
version: '{branch}-{build}'
build: off
cache:
- '%LOCALAPPDATA%\pip\Cache'
environment:
global:
WITH_COMPILER: 'cmd /E:ON /V:ON /C .\ci\appveyor-with-compiler.cmd'
matrix:
- TOXENV: check
PYTHON_HOME: C:\Python27
PYTHON_VERSION: '2.7'
PYTHON_ARCH: '32'
- TOXENV: 'py27,codecov'
TOXPYTHON: C:\Python27\python.exe
PYTHON_HOME: C:\Python27
PYTHON_VERSION: '2.7'
PYTHON_ARCH: '32'
- TOXENV: 'py27,codecov'
TOXPYTHON: C:\Python27-x64\python.exe
WINDOWS_SDK_VERSION: v7.0
PYTHON_HOME: C:\Python27-x64
PYTHON_VERSION: '2.7'
PYTHON_ARCH: '64'
- TOXENV: 'py34,codecov'
TOXPYTHON: C:\Python34\python.exe
PYTHON_HOME: C:\Python34
PYTHON_VERSION: '3.4'
PYTHON_ARCH: '32'
- TOXENV: 'py34,codecov'
TOXPYTHON: C:\Python34-x64\python.exe
WINDOWS_SDK_VERSION: v7.1
PYTHON_HOME: C:\Python34-x64
PYTHON_VERSION: '3.4'
PYTHON_ARCH: '64'
- TOXENV: 'py35,codecov'
TOXPYTHON: C:\Python35\python.exe
PYTHON_HOME: C:\Python35
PYTHON_VERSION: '3.5'
PYTHON_ARCH: '32'
- TOXENV: 'py35,codecov'
TOXPYTHON: C:\Python35-x64\python.exe
PYTHON_HOME: C:\Python35-x64
PYTHON_VERSION: '3.5'
PYTHON_ARCH: '64'
init:
- ps: echo $env:TOXENV
- ps: ls C:\Python*
install:
- python -u ci\appveyor-bootstrap.py
- '%PYTHON_HOME%\Scripts\virtualenv --version'
- '%PYTHON_HOME%\Scripts\easy_install --version'
- '%PYTHON_HOME%\Scripts\pip --version'
- '%PYTHON_HOME%\Scripts\tox --version'
test_script:
- '%WITH_COMPILER% %PYTHON_HOME%\Scripts\tox'
on_failure:
- ps: dir "env:"
- ps: get-content .tox\*\log\*
artifacts:
- path: dist\*
### To enable remote debugging uncomment this (also, see: http://www.appveyor.com/docs/how-to/rdp-to-build-worker):
# on_finish:
# - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

115
ci/appveyor-bootstrap.py Normal file
Просмотреть файл

@ -0,0 +1,115 @@
"""
AppVeyor will at least have few Pythons around so there's no point of implementing a bootstrapper in PowerShell.
This is a port of https://github.com/pypa/python-packaging-user-guide/blob/master/source/code/install.ps1
with various fixes and improvements that just weren't feasible to implement in PowerShell.
"""
from __future__ import print_function
from os import environ
from os.path import exists
from subprocess import CalledProcessError
from subprocess import check_call
try:
from urllib.request import urlretrieve
except ImportError:
from urllib import urlretrieve
BASE_URL = "https://www.python.org/ftp/python/"
GET_PIP_URL = "https://bootstrap.pypa.io/get-pip.py"
GET_PIP_PATH = "C:\get-pip.py"
URLS = {
("2.7", "64"): BASE_URL + "2.7.10/python-2.7.10.amd64.msi",
("2.7", "32"): BASE_URL + "2.7.10/python-2.7.10.msi",
# NOTE: no .msi installer for 3.3.6
("3.3", "64"): BASE_URL + "3.3.3/python-3.3.3.amd64.msi",
("3.3", "32"): BASE_URL + "3.3.3/python-3.3.3.msi",
("3.4", "64"): BASE_URL + "3.4.3/python-3.4.3.amd64.msi",
("3.4", "32"): BASE_URL + "3.4.3/python-3.4.3.msi",
("3.5", "64"): BASE_URL + "3.5.0/python-3.5.0-amd64.exe",
("3.5", "32"): BASE_URL + "3.5.0/python-3.5.0.exe",
}
INSTALL_CMD = {
# Commands are allowed to fail only if they are not the last command. Eg: uninstall (/x) allowed to fail.
"2.7": [["msiexec.exe", "/L*+!", "install.log", "/qn", "/x", "{path}"],
["msiexec.exe", "/L*+!", "install.log", "/qn", "/i", "{path}", "TARGETDIR={home}"]],
"3.3": [["msiexec.exe", "/L*+!", "install.log", "/qn", "/x", "{path}"],
["msiexec.exe", "/L*+!", "install.log", "/qn", "/i", "{path}", "TARGETDIR={home}"]],
"3.4": [["msiexec.exe", "/L*+!", "install.log", "/qn", "/x", "{path}"],
["msiexec.exe", "/L*+!", "install.log", "/qn", "/i", "{path}", "TARGETDIR={home}"]],
"3.5": [["{path}", "/quiet", "TargetDir={home}"]],
}
def download_file(url, path):
print("Downloading: {} (into {})".format(url, path))
progress = [0, 0]
def report(count, size, total):
progress[0] = count * size
if progress[0] - progress[1] > 1000000:
progress[1] = progress[0]
print("Downloaded {:,}/{:,} ...".format(progress[1], total))
dest, _ = urlretrieve(url, path, reporthook=report)
return dest
def install_python(version, arch, home):
print("Installing Python", version, "for", arch, "bit architecture to", home)
if exists(home):
return
path = download_python(version, arch)
print("Installing", path, "to", home)
success = False
for cmd in INSTALL_CMD[version]:
cmd = [part.format(home=home, path=path) for part in cmd]
print("Running:", " ".join(cmd))
try:
check_call(cmd)
except CalledProcessError as exc:
print("Failed command", cmd, "with:", exc)
if exists("install.log"):
with open("install.log") as fh:
print(fh.read())
else:
success = True
if success:
print("Installation complete!")
else:
print("Installation failed")
def download_python(version, arch):
for _ in range(3):
try:
return download_file(URLS[version, arch], "installer.exe")
except Exception as exc:
print("Failed to download:", exc)
print("Retrying ...")
def install_pip(home):
pip_path = home + "/Scripts/pip.exe"
python_path = home + "/python.exe"
if exists(pip_path):
print("pip already installed.")
else:
print("Installing pip...")
download_file(GET_PIP_URL, GET_PIP_PATH)
print("Executing:", python_path, GET_PIP_PATH)
check_call([python_path, GET_PIP_PATH])
def install_packages(home, *packages):
cmd = [home + "/Scripts/pip.exe", "install"]
cmd.extend(packages)
check_call(cmd)
if __name__ == "__main__":
install_python(environ['PYTHON_VERSION'], environ['PYTHON_ARCH'], environ['PYTHON_HOME'])
install_pip(environ['PYTHON_HOME'])
install_packages(environ['PYTHON_HOME'], "setuptools>=18.0.1", "wheel", "tox", "virtualenv>=13.1.0")

109
ci/appveyor-download.py Executable file
Просмотреть файл

@ -0,0 +1,109 @@
#!/usr/bin/env python
"""
Use the AppVeyor API to download Windows artifacts.
Taken from: https://bitbucket.org/ned/coveragepy/src/tip/ci/download_appveyor.py
# Licensed under the Apache License: http://www.apache.org/licenses/LICENSE-2.0
# For details: https://bitbucket.org/ned/coveragepy/src/default/NOTICE.txt
"""
from __future__ import unicode_literals
import argparse
import os
import zipfile
import requests
def make_auth_headers():
"""Make the authentication headers needed to use the Appveyor API."""
path = os.path.expanduser("~/.appveyor.token")
if not os.path.exists(path):
raise RuntimeError(
"Please create a file named `.appveyor.token` in your home directory. "
"You can get the token from https://ci.appveyor.com/api-token"
)
with open(path) as f:
token = f.read().strip()
headers = {
'Authorization': 'Bearer {}'.format(token),
}
return headers
def download_latest_artifacts(account_project, build_id):
"""Download all the artifacts from the latest build."""
if build_id is None:
url = "https://ci.appveyor.com/api/projects/{}".format(account_project)
else:
url = "https://ci.appveyor.com/api/projects/{}/build/{}".format(account_project, build_id)
build = requests.get(url, headers=make_auth_headers()).json()
jobs = build['build']['jobs']
print(u"Build {0[build][version]}, {1} jobs: {0[build][message]}".format(build, len(jobs)))
for job in jobs:
name = job['name']
print(u" {0}: {1[status]}, {1[artifactsCount]} artifacts".format(name, job))
url = "https://ci.appveyor.com/api/buildjobs/{}/artifacts".format(job['jobId'])
response = requests.get(url, headers=make_auth_headers())
artifacts = response.json()
for artifact in artifacts:
is_zip = artifact['type'] == "Zip"
filename = artifact['fileName']
print(u" {0}, {1} bytes".format(filename, artifact['size']))
url = "https://ci.appveyor.com/api/buildjobs/{}/artifacts/{}".format(job['jobId'], filename)
download_url(url, filename, make_auth_headers())
if is_zip:
unpack_zipfile(filename)
os.remove(filename)
def ensure_dirs(filename):
"""Make sure the directories exist for `filename`."""
dirname = os.path.dirname(filename)
if dirname and not os.path.exists(dirname):
os.makedirs(dirname)
def download_url(url, filename, headers):
"""Download a file from `url` to `filename`."""
ensure_dirs(filename)
response = requests.get(url, headers=headers, stream=True)
if response.status_code == 200:
with open(filename, 'wb') as f:
for chunk in response.iter_content(16 * 1024):
f.write(chunk)
else:
print(u" Error downloading {}: {}".format(url, response))
def unpack_zipfile(filename):
"""Unpack a zipfile, using the names in the zip."""
with open(filename, 'rb') as fzip:
z = zipfile.ZipFile(fzip)
for name in z.namelist():
print(u" extracting {}".format(name))
ensure_dirs(name)
z.extract(name)
parser = argparse.ArgumentParser(description='Download artifacts from AppVeyor.')
parser.add_argument('--id',
metavar='PROJECT_ID',
default='mozilla/build-mar',
help='Project ID in AppVeyor.')
parser.add_argument('build',
nargs='?',
metavar='BUILD_ID',
help='Build ID in AppVeyor. Eg: master-123')
if __name__ == "__main__":
# import logging
# logging.basicConfig(level="DEBUG")
args = parser.parse_args()
download_latest_artifacts(args.id, args.build)

Просмотреть файл

@ -0,0 +1,46 @@
:: To build extensions for 64 bit Python 3, we need to configure environment
:: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 4 (SDK v7.1)
::
:: To build extensions for 64 bit Python 2, we need to configure environment
:: variables to use the MSVC 2008 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 3.5 (SDK v7.0)
::
:: 32 bit builds do not require specific environment configurations.
::
:: Note: this script needs to be run with the /E:ON and /V:ON flags for the
:: cmd interpreter, at least for (SDK v7.0)
::
:: More details at:
:: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
:: http://stackoverflow.com/a/13751649/163740
::
:: Author: Olivier Grisel
:: License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
SET COMMAND_TO_RUN=%*
SET WIN_SDK_ROOT=C:\Program Files\Microsoft SDKs\Windows
SET WIN_WDK="c:\Program Files (x86)\Windows Kits\10\Include\wdf"
ECHO SDK: %WINDOWS_SDK_VERSION% ARCH: %PYTHON_ARCH%
IF "%PYTHON_VERSION%"=="3.5" (
IF EXIST %WIN_WDK% (
REM See: https://connect.microsoft.com/VisualStudio/feedback/details/1610302/
REN %WIN_WDK% 0wdf
)
GOTO main
)
IF "%PYTHON_ARCH%"=="32" (
GOTO main
)
SET DISTUTILS_USE_SDK=1
SET MSSdk=1
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Setup\WindowsSdkVer.exe" -q -version:%WINDOWS_SDK_VERSION%
CALL "%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Bin\SetEnv.cmd" /x64 /release
:main
ECHO Executing: %COMMAND_TO_RUN%
CALL %COMMAND_TO_RUN% || EXIT 1

57
ci/bootstrap.py Executable file
Просмотреть файл

@ -0,0 +1,57 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, print_function, unicode_literals
import os
import sys
from os.path import abspath
from os.path import dirname
from os.path import exists
from os.path import join
if __name__ == "__main__":
base_path = dirname(dirname(abspath(__file__)))
print("Project path: {0}".format(base_path))
env_path = join(base_path, ".tox", "bootstrap")
if sys.platform == "win32":
bin_path = join(env_path, "Scripts")
else:
bin_path = join(env_path, "bin")
if not exists(env_path):
import subprocess
print("Making bootstrap env in: {0} ...".format(env_path))
try:
subprocess.check_call(["virtualenv", env_path])
except subprocess.CalledProcessError:
subprocess.check_call([sys.executable, "-m", "virtualenv", env_path])
print("Installing `jinja2` into bootstrap environment...")
subprocess.check_call([join(bin_path, "pip"), "install", "jinja2"])
activate = join(bin_path, "activate_this.py")
# noinspection PyCompatibility
exec(compile(open(activate, "rb").read(), activate, "exec"), dict(__file__=activate))
import jinja2
import subprocess
jinja = jinja2.Environment(
loader=jinja2.FileSystemLoader(join(base_path, "ci", "templates")),
trim_blocks=True,
lstrip_blocks=True,
keep_trailing_newline=True
)
tox_environments = [
line.strip()
# WARNING: 'tox' must be installed globally or in the project's virtualenv
for line in subprocess.check_output(['tox', '--listenvs'], universal_newlines=True).splitlines()
]
tox_environments = [line for line in tox_environments if line not in ['clean', 'report', 'docs', 'check']]
for name in os.listdir(join("ci", "templates")):
with open(join(base_path, name), "w") as fh:
fh.write(jinja.get_template(name).render(tox_environments=tox_environments))
print("Wrote {}".format(name))
print("DONE.")

38
ci/templates/.travis.yml Normal file
Просмотреть файл

@ -0,0 +1,38 @@
language: python
python: '3.5'
sudo: false
env:
global:
- LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so
- SEGFAULT_SIGNALS=all
matrix:
- TOXENV=check
- TOXENV=docs
{% for env in tox_environments %}{{ '' }}
- TOXENV={{ env }},coveralls,codecov
{% endfor %}
before_install:
- python --version
- uname -a
- lsb_release -a
install:
- pip install tox
- virtualenv --version
- easy_install --version
- pip --version
- tox --version
script:
- tox -v
after_failure:
- more .tox/log/* | cat
- more .tox/*/log/* | cat
before_cache:
- rm -rf $HOME/.cache/pip/log
cache:
directories:
- $HOME/.cache/pip
notifications:
email:
on_success: never
on_failure: always

53
ci/templates/appveyor.yml Normal file
Просмотреть файл

@ -0,0 +1,53 @@
version: '{branch}-{build}'
build: off
cache:
- '%LOCALAPPDATA%\pip\Cache'
environment:
global:
WITH_COMPILER: 'cmd /E:ON /V:ON /C .\ci\appveyor-with-compiler.cmd'
matrix:
- TOXENV: check
PYTHON_HOME: C:\Python27
PYTHON_VERSION: '2.7'
PYTHON_ARCH: '32'
{% for env in tox_environments %}{% if env.startswith(('py27', 'py34', 'py35')) %}
- TOXENV: '{{ env }},codecov'
TOXPYTHON: C:\Python{{ env[2:4] }}\python.exe
PYTHON_HOME: C:\Python{{ env[2:4] }}
PYTHON_VERSION: '{{ env[2] }}.{{ env[3] }}'
PYTHON_ARCH: '32'
- TOXENV: '{{ env }},codecov'
TOXPYTHON: C:\Python{{ env[2:4] }}-x64\python.exe
{%- if env.startswith(('py2', 'py33', 'py34')) %}
WINDOWS_SDK_VERSION: v7.{{ '1' if env.startswith('py3') else '0' }}
{%- endif %}
PYTHON_HOME: C:\Python{{ env[2:4] }}-x64
PYTHON_VERSION: '{{ env[2] }}.{{ env[3] }}'
PYTHON_ARCH: '64'
{% endif %}{% endfor %}
init:
- ps: echo $env:TOXENV
- ps: ls C:\Python*
install:
- python -u ci\appveyor-bootstrap.py
- '%PYTHON_HOME%\Scripts\virtualenv --version'
- '%PYTHON_HOME%\Scripts\easy_install --version'
- '%PYTHON_HOME%\Scripts\pip --version'
- '%PYTHON_HOME%\Scripts\tox --version'
test_script:
- '%WITH_COMPILER% %PYTHON_HOME%\Scripts\tox'
on_failure:
- ps: dir "env:"
- ps: get-content .tox\*\log\*
artifacts:
- path: dist\*
### To enable remote debugging uncomment this (also, see: http://www.appveyor.com/docs/how-to/rdp-to-build-worker):
# on_finish:
# - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

1
docs/authors.rst Normal file
Просмотреть файл

@ -0,0 +1 @@
.. include:: ../AUTHORS.rst

1
docs/changelog.rst Normal file
Просмотреть файл

@ -0,0 +1 @@
.. include:: ../CHANGELOG.rst

53
docs/conf.py Normal file
Просмотреть файл

@ -0,0 +1,53 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
import os
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.autosummary',
'sphinx.ext.coverage',
'sphinx.ext.doctest',
'sphinx.ext.extlinks',
'sphinx.ext.ifconfig',
'sphinx.ext.napoleon',
'sphinx.ext.todo',
'sphinx.ext.viewcode',
]
if os.getenv('SPELLCHECK'):
extensions += 'sphinxcontrib.spelling',
spelling_show_suggestions = True
spelling_lang = 'en_US'
source_suffix = '.rst'
master_doc = 'index'
project = u'mar'
year = '2016'
author = u'Chris AtLee'
copyright = '{0}, {1}'.format(year, author)
version = release = u'2.0'
pygments_style = 'trac'
templates_path = ['.']
extlinks = {
'issue': ('https://github.com/mozilla/build-mar/issues/%s', '#'),
'pr': ('https://github.com/mozilla/build-mar/pull/%s', 'PR #'),
}
# on_rtd is whether we are on readthedocs.org
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
if not on_rtd: # only set the theme if we're building docs locally
html_theme = 'sphinx_rtd_theme'
html_use_smartypants = True
html_last_updated_fmt = '%b %d, %Y'
html_split_index = False
html_sidebars = {
'**': ['searchbox.html', 'globaltoc.html', 'sourcelink.html'],
}
html_short_title = '%s-%s' % (project, version)
napoleon_use_ivar = True
napoleon_use_rtype = False
napoleon_use_param = False

1
docs/contributing.rst Normal file
Просмотреть файл

@ -0,0 +1 @@
.. include:: ../CONTRIBUTING.rst

22
docs/index.rst Normal file
Просмотреть файл

@ -0,0 +1,22 @@
========
Contents
========
.. toctree::
:maxdepth: 2
readme
installation
usage
reference/index
contributing
authors
changelog
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

7
docs/installation.rst Normal file
Просмотреть файл

@ -0,0 +1,7 @@
============
Installation
============
At the command line::
pip install mardor

1
docs/readme.rst Normal file
Просмотреть файл

@ -0,0 +1 @@
.. include:: ../README.rst

7
docs/reference/index.rst Normal file
Просмотреть файл

@ -0,0 +1,7 @@
Reference
=========
.. toctree::
:glob:
mardor*

Просмотреть файл

@ -0,0 +1,9 @@
mardor
======
.. testsetup::
from mardor import *
.. automodule:: mardor
:members:

3
docs/requirements.txt Normal file
Просмотреть файл

@ -0,0 +1,3 @@
sphinx>=1.3
sphinx-rtd-theme
-e .

Просмотреть файл

@ -0,0 +1,11 @@
builtin
builtins
classmethod
staticmethod
classmethods
staticmethods
args
kwargs
callstack
Changelog
Indices

7
docs/usage.rst Normal file
Просмотреть файл

@ -0,0 +1,7 @@
=====
Usage
=====
To use mar in a project::
import mardor

35
setup.cfg Normal file
Просмотреть файл

@ -0,0 +1,35 @@
[bdist_wheel]
universal = 1
[flake8]
max-line-length = 140
exclude = tests/*,*/migrations/*,*/south_migrations/*
[tool:pytest]
norecursedirs =
.git
.tox
.env
dist
build
south_migrations
migrations
python_files =
test_*.py
*_test.py
tests.py
addopts =
-rxEfsw
--strict
--doctest-modules
--doctest-glob=\*.rst
--tb=short
[isort]
force_single_line=True
line_length=120
known_first_party=mardor
default_section=THIRDPARTY
forced_separate=test_mardor
not_skip = __init__.py
skip = migrations, south_migrations

Просмотреть файл

@ -1,22 +1,87 @@
#!/usr/bin/env python
# -*- encoding: utf-8 -*-
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
from setuptools import setup, find_packages
from __future__ import absolute_import
from __future__ import print_function
import io
import re
from glob import glob
from os.path import basename
from os.path import dirname
from os.path import join
from os.path import splitext
from setuptools import find_packages
from setuptools import setup
def read(*names, **kwargs):
return io.open(
join(dirname(__file__), *names),
encoding=kwargs.get('encoding', 'utf8')
).read()
setup(
name="mar",
version="2.0pre",
author="Chris AtLee",
author_email="catlee@mozilla.com",
packages=find_packages(),
url="https://github.com/mozilla/build-mar",
license="MPL 2.0",
description="MAR (Mozilla ARchive) Python implementation",
install_requires=['cryptography', 'construct'],
long_description=open('README.md').read(),
name='mar',
version='2.0',
license='MPL 2.0',
description='Package for handling Mozilla Archive files.',
long_description='%s\n%s' % (
re.compile('^.. start-badges.*^.. end-badges', re.M | re.S).sub('', read('README.rst')),
re.sub(':[a-z]+:`~?(.*?)`', r'``\1``', read('CHANGELOG.rst'))
),
author='Chris AtLee',
author_email='catlee@mozilla.com',
url='https://github.com/mozilla/build-mar',
packages=find_packages('src'),
package_dir={'': 'src'},
py_modules=[splitext(basename(path))[0] for path in glob('src/*.py')],
include_package_data=True,
zip_safe=False,
classifiers=[
# complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)',
'Operating System :: Unix',
'Operating System :: POSIX',
'Operating System :: Microsoft :: Windows',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: Implementation :: CPython',
'Programming Language :: Python :: Implementation :: PyPy',
# uncomment if you test on these interpreters:
# 'Programming Language :: Python :: Implementation :: IronPython',
# 'Programming Language :: Python :: Implementation :: Jython',
# 'Programming Language :: Python :: Implementation :: Stackless',
'Topic :: Utilities',
],
keywords=[
'mozilla', 'mar', 'archive',
],
install_requires=[
'click',
'cryptography',
'construct',
'backports.lzma',
],
extras_require={
# eg:
# 'rst': ['docutils>=0.11'],
# ':python_version=="2.6"': ['argparse'],
},
entry_points={
'console_scripts': [
'mar = mardor.cli:main',
],
]
},
)

Просмотреть файл

Просмотреть файл

@ -4,15 +4,16 @@
# The MAR format is documented at
# https://wiki.mozilla.org/Software_Update:MAR
import logging
import os
import sys
from argparse import ArgumentParser, REMAINDER
from argparse import REMAINDER
from argparse import ArgumentParser
import mardor.mozilla
from mardor.reader import MarReader
from mardor.writer import MarWriter
import mardor.mozilla
import logging
log = logging.getLogger(__name__)

Просмотреть файл

@ -8,9 +8,21 @@ This relies on the construct module for specifying the data structures.
See also https://wiki.mozilla.org/Software_Update:MAR
"""
from construct import (CString, Struct, Array, Bytes, Const, GreedyRange, If,
Int32ub, Int64ub, Pointer, this, Rebuild,
len_, Padding, Select)
from construct import Array
from construct import Bytes
from construct import Const
from construct import CString
from construct import GreedyRange
from construct import If
from construct import Int32ub
from construct import Int64ub
from construct import Padding
from construct import Pointer
from construct import Rebuild
from construct import Select
from construct import Struct
from construct import len_
from construct import this
mar_header = "mar_header" / Struct(
"magic" / Const(b"MAR1"),

Просмотреть файл

@ -67,4 +67,3 @@ ikFkUzNDBbwTp3CLCcvR40GZdkQ2XfjFNZmlhmH6iJYmRwDT4SRnAiicdnDcK+o/
alRnlvBZWbO9ZoiXbyuxXjZRRRx6vO8UTEOQTsKmXBAGZCW6z0+AAlgvPnILgOG+
jQIDAQAB
-----END PUBLIC KEY-----"""

Просмотреть файл

@ -8,16 +8,20 @@ verify MAR files.
"""
import os
from enum import Enum
from cryptography.exceptions import InvalidSignature
from mardor.utils import (file_iter, takeexactly, auto_decompress_stream,
bz2_decompress_stream, write_to_file, mkdir,
safejoin)
from mardor.format import mar
from mardor.signing import get_signature_data, make_verifier_v1
from mardor.signing import get_signature_data
from mardor.signing import make_verifier_v1
from mardor.utils import auto_decompress_stream
from mardor.utils import bz2_decompress_stream
from mardor.utils import file_iter
from mardor.utils import mkdir
from mardor.utils import safejoin
from mardor.utils import takeexactly
from mardor.utils import write_to_file
class Decompression(Enum):

Просмотреть файл

@ -2,11 +2,13 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
"""Signing, verification and key support for MAR files."""
from construct import Int32ub
from construct import Int64ub
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import padding, rsa
from cryptography.hazmat.primitives import hashes, serialization
from construct import Int64ub, Int32ub
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.asymmetric import rsa
from mardor.format import sigs_header
from mardor.utils import file_iter

Просмотреть файл

@ -2,8 +2,8 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
"""Utilities for reading/writing MAR files."""
import os
import bz2
import os
from functools import partial
from itertools import chain

Просмотреть файл

@ -6,14 +6,17 @@
This module provides the MarWriter class which is used to write MAR files.
"""
import os
from mardor.utils import bz2_compress_stream, write_to_file
from mardor.format import mar_header, sigs_header, extras_header, index_header
from mardor.signing import make_signer_v1, get_signature_data
from enum import Enum
from mardor.format import extras_header
from mardor.format import index_header
from mardor.format import mar_header
from mardor.format import sigs_header
from mardor.signing import get_signature_data
from mardor.signing import make_signer_v1
from mardor.utils import bz2_compress_stream
from mardor.utils import write_to_file
class Compression(Enum):
"""

Просмотреть файл

@ -1,4 +1,6 @@
pytest
pytest-cov
pytest-travis-fold
hypothesis
mock
-rrequirements.txt

Просмотреть файл

@ -4,11 +4,23 @@
#
# pip-compile --output-file test-requirements.txt test-requirements.in
#
backports.lzma==0.0.6
cffi==1.8.3
construct==2.8.8
coverage==4.2 # via pytest-cov
cryptography==1.5.2
enum34==1.1.6
hypothesis==3.6.0
idna==2.1
mock==2.0.0
pbr==1.10.0 # via mock
py==1.4.31 # via pytest
pyasn1==0.1.9
pycparser==2.17
pytest-cov==2.4.0
pytest-travis-fold==1.2.0
pytest==3.0.3
six==1.10.0 # via mock
six==1.10.0
# The following packages are considered to be unsafe in a requirements file:
# setuptools # via cryptography

Просмотреть файл

@ -2,11 +2,14 @@
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
import os
from mardor import cli
from mardor.reader import Decompression, MarReader
from mardor import mozilla
from pytest import fixture, raises
from pytest import fixture
from pytest import raises
from mardor import cli
from mardor import mozilla
from mardor.reader import Decompression
from mardor.reader import MarReader
TEST_MAR = os.path.join(os.path.dirname(__file__), 'test.mar')

Просмотреть файл

@ -1,9 +1,11 @@
import os
from mardor.mozilla import (release1, release2,
nightly1, nightly2,
dep1, dep2)
from mardor.mozilla import dep1
from mardor.mozilla import dep2
from mardor.mozilla import nightly1
from mardor.mozilla import nightly2
from mardor.mozilla import release1
from mardor.mozilla import release2
from mardor.reader import MarReader

Просмотреть файл

@ -1,8 +1,8 @@
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
import os
import bz2
import os
import pytest

Просмотреть файл

@ -3,13 +3,16 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
from itertools import repeat
import pytest
from mardor.utils import (takeexactly, bz2_compress_stream,
bz2_decompress_stream, auto_decompress_stream, mkdir)
from hypothesis import given, assume
import hypothesis.strategies as st
import pytest
from hypothesis import assume
from hypothesis import given
from mardor.utils import auto_decompress_stream
from mardor.utils import bz2_compress_stream
from mardor.utils import bz2_decompress_stream
from mardor.utils import mkdir
from mardor.utils import takeexactly
@given(st.lists(st.binary()), st.integers(min_value=0))

Просмотреть файл

@ -3,9 +3,9 @@
# file, You can obtain one at http://mozilla.org/MPL/2.0/.
import bz2
from mardor.writer import MarWriter
from mardor.reader import MarReader
from mardor.signing import make_rsa_keypair
from mardor.writer import MarWriter
def test_writer(tmpdir):

117
tox.ini
Просмотреть файл

@ -1,41 +1,106 @@
; a generative tox configuration, see: https://testrun.org/tox/latest/config.html#generative-envlist
[tox]
envlist = cleanup,py27,py35,flake8,html,bandit
envlist =
clean,
check,
{py27,py34,py35,py36,pypy},
report,
docs
[testenv]
usedevelop = True
passenv = TRAVIS TRAVIS_JOB_ID TRAVIS_BRANCH
basepython =
pypy: {env:TOXPYTHON:pypy}
{py27,docs,spell}: {env:TOXPYTHON:python2.7}
py34: {env:TOXPYTHON:python3.4}
py35: {env:TOXPYTHON:python3.5}
py36: {env:TOXPYTHON:python3.6}
{clean,check,report,coveralls,codecov}: python3.5
bootstrap: python
setenv =
PYTHONPATH = {toxinidir}
PYTHONPATH={toxinidir}/tests
PYTHONUNBUFFERED=yes
passenv =
*
usedevelop = false
deps =
-rtest-requirements.txt
-rrequirements.txt
commands =
pytest --cov=mardor --cov-append tests/ {posargs}
{posargs:py.test --cov --cov-report=term-missing -vv tests}
[testenv:flake8]
[testenv:bootstrap]
deps =
jinja2
matrix
skip_install = true
commands =
python ci/bootstrap.py
passenv =
*
[testenv:spell]
setenv =
SPELLCHECK=1
commands =
sphinx-build -b spelling docs dist/docs
skip_install = true
deps =
-r{toxinidir}/docs/requirements.txt
sphinxcontrib-spelling
pyenchant
[testenv:docs]
deps =
-r{toxinidir}/docs/requirements.txt
commands =
sphinx-build {posargs:-E} -b html docs dist/docs
sphinx-build -b linkcheck docs dist/docs
[testenv:check]
deps =
docutils
check-manifest
flake8
flake8-docstrings
skip_install = True
usedevelop = False
commands = flake8 mardor
readme-renderer
pygments
isort
skip_install = true
commands =
python setup.py check --strict --metadata --restructuredtext
check-manifest {toxinidir}
flake8 src tests setup.py
isort --verbose --check-only --diff --recursive src tests setup.py
[testenv:cleanup]
[testenv:coveralls]
deps =
coveralls
skip_install = true
commands =
coverage combine --append
coverage report
coveralls []
[testenv:codecov]
deps =
codecov
skip_install = true
commands =
coverage combine --append
coverage report
coverage xml --ignore-errors
codecov []
[testenv:report]
deps = coverage
skip_install = True
usedevelop = False
skip_install = true
commands =
coverage combine --append
coverage report
coverage html
[testenv:clean]
commands = coverage erase
[testenv:html]
skip_install = true
deps = coverage
skip_install = True
usedevelop = False
commands = coverage html -i
[testenv:bandit]
deps = bandit==1.2.0
skip_install = True
usedevelop = False
commands = bandit -r mardor