Sync Nextcloud releases with GitHub (#544)

* add command for fetching nextcloud releases from github

* simplify loop

* pull out client

* update allauth

* add test for parsing github json

* add docs, finish command

* fix style

* remove extra file

* delete file

* narrow down fixtures

* update changelog

* update changelog
This commit is contained in:
Bernhard Posselt 2018-02-04 20:57:47 +01:00 коммит произвёл GitHub
Родитель 4ca424dac8
Коммит de6925e46f
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
17 изменённых файлов: 884 добавлений и 5 удалений

1
.gitignore поставляемый
Просмотреть файл

@ -15,6 +15,7 @@ venv
.idea/tasks.xml
.idea/codeStyleSettings.xml
.idea/inspectionProfiles
.idea/codeStyles
nextcloudappstore/local_settings.py
*.sqlite3
*.wp[ru]

Просмотреть файл

@ -1,6 +1,18 @@
# Changelog
## [Unreleased]
## [3.2.0] - 2018-02-04
### Added
- **syncnextcloudreleases** to sync Nextcloud releases directly from GitHub
### Removed
- Nextcloud releases are not imported via fixtures anymore, use the **syncnextcloudreleases** command
### Security
- Narrow down fixtures to not accidentally import test data on production systems. Check if a user with the user name **admin** was created. If so delete that user from your system.
## [3.1.2] - 2018-02-02

Просмотреть файл

@ -58,7 +58,7 @@ endif
.PHONY: initdb
initdb:
$(manage) migrate --settings nextcloudappstore.settings.development
$(manage) loaddata $(CURDIR)/nextcloudappstore/**/fixtures/*.json --settings nextcloudappstore.settings.development
$(manage) loaddata $(CURDIR)/nextcloudappstore/core/fixtures/*.json --settings nextcloudappstore.settings.development
$(manage) createsuperuser --username admin --email admin@admin.com --noinput --settings nextcloudappstore.settings.development
$(manage) verifyemail --username admin --email admin@admin.com --settings nextcloudappstore.settings.development
$(manage) setdefaultadminpassword --settings nextcloudappstore.settings.development

Просмотреть файл

@ -42,6 +42,7 @@ Look here if you want to install the store on your server and keep it up to date
prodinstall
prodinstalldocker
upgradenotices
App Store Developer Documentation
---------------------------------

Просмотреть файл

@ -231,7 +231,7 @@ Loading Initial Data
--------------------
To pre-populate the database with categories and other data run the following command::
python manage.py loaddata nextcloudappstore/**/fixtures/*.json
python manage.py loaddata nextcloudappstore/core/fixtures/*.json
Initializing Translations
-------------------------
@ -365,6 +365,47 @@ Afterwards your **client id** and **client secret** are displayed. These need to
.. note:: For local testing use localhost:8000 as domain name. Furthermore the confirmation mail will also be printed in your shell that was used to start the development server.
.. _prod_install_release_sync:
Sync Nextcloud Releases from GitHub
-----------------------------------
The App Store needs to know about Nextcloud versions because:
* app releases are grouped by app version on the app detail page
* you can :ref:`access a REST API to get all available versions <api-all-platforms>`
Before **3.2.0** releases were imported either manually or via the a shipped JSON file. This process proved to be very tedious. In **3.2.0** a command was introduced to sync releases (git tags) directly from GitHub.
You can run the command by giving it the oldest supported Nextcloud version::
python manage.py syncnextcloudreleases --oldest-supported="12.0.0"
All existing versions prior to this release will be marked as not having a release, new versions will be imported and the latest version will be marked as current version.
You can also do a test run and see what kind of versions would be imported::
python manage.py syncnextcloudreleases --oldest-supported="12.0.0" --print
The GitHub API is rate limited to 60 requests per second. Depending on how far back your **oldest-supported** version goes a single command might fetch multiple pages of releases. If you want to run the command more than 10 times per hour it is recommended to `obtain and configure a GitHub OAuth2 token <https://help.github.com/articles/git-automation-with-oauth-tokens/>`_.
After obtaining the token from GitHub, add it anywhere in your settings file (**nextcloudappstore/settings/production.py**), e.g.:
.. code-block:: python
GITHUB_API_TOKEN = '4bab6b3dfeds8857371a48855d3e87d38d4b7e65'
To automate syncing you might want to add the command as a cronjob and schedule it every hour.
.. note:: Only one sync command should be run at a time, otherwise race conditions might cause unpredictable results. To ensure this use a proper cronjob daemon that supports running only one command at a time, for instance `SystemD timers <https://wiki.archlinux.org/index.php/Systemd/Timers>`_
.. note:: If run the command outside of your virtual environment you need to prefix the full path to the desired Python executable, e.g.
::
venv/bin/python manage.py syncnextcloudreleases --oldest-supported="12.0.0"
Keeping Up To Date
------------------
Updating an instance is scripted in **scripts/maintenance/update.sh**. Depending on your distribution you will have to adjust the scripts contents.

Просмотреть файл

@ -388,3 +388,39 @@ Afterwards your **client id** and **client secret** are displayed. These need to
.. note:: The above mentioned domains need to be changed if you want to run the App Store on a different server.
.. note:: For local testing use localhost:8000 as domain name. Furthermore the confirmation mail will also be printed in your shell that was used to start the development server.
.. _prod_install_release_sync_docker:
Sync Nextcloud Releases from GitHub
-----------------------------------
The App Store needs to know about Nextcloud versions because:
* app releases are grouped by app version on the app detail page
* you can :ref:`access a REST API to get all available versions <api-all-platforms>`
Before **3.2.0** releases were imported either manually or via the a shipped JSON file. This process proved to be very tedious. In **3.2.0** a command was introduced to sync releases (git tags) directly from GitHub.
You can run the command by giving it the oldest supported Nextcloud version::
sudo docker-compose exec python manage.py syncnextcloudreleases --oldest-supported="12.0.0"
All existing versions prior to this release will be marked as not having a release, new versions will be imported and the latest version will be marked as current version.
You can also do a test run and see what kind of versions would be imported::
sudo docker-compose exec python manage.py syncnextcloudreleases --oldest-supported="12.0.0" --print
The GitHub API is rate limited to 60 requests per second. Depending on how far back your **oldest-supported** version goes a single command might fetch multiple pages of releases. If you want to run the command more than 10 times per hour it is recommended to `obtain and configure a GitHub OAuth2 token <https://help.github.com/articles/git-automation-with-oauth-tokens/>`_.
After obtaining the token from GitHub, add it anywhere in your settings file (**production.py**), e.g.:
.. code-block:: python
GITHUB_API_TOKEN = '4bab6b3dfeds8857371a48855d3e87d38d4b7e65'
To automate syncing you might want to add the command as a cronjob and schedule it every hour.
.. note:: Only one sync command should be run at a time, otherwise race conditions might cause unpredictable results. To ensure this use a proper cronjob daemon that supports running only one command at a time, for instance `SystemD timers <https://wiki.archlinux.org/index.php/Systemd/Timers>`_

15
docs/upgradenotices.rst Normal file
Просмотреть файл

@ -0,0 +1,15 @@
=====================
Store Upgrade Notices
=====================
3.2.0
=====
Configuring Nextcloud Release Sync
----------------------------------
**3.2.0** changed the way Nextcloud releases are managed in the App Store. Instead of manually adjusting releases in the admin interface or importing updated fixtures via JSON files releases are now synced directly from GitHub.
Consult :ref:`prod_install_release_sync` or if you are using Docker :ref:`prod_install_release_sync_docker` in order to run or configure the release sync command.

Просмотреть файл

@ -0,0 +1,112 @@
from itertools import chain, takewhile
from typing import Iterable, List
import requests
from semantic_version import Version
from nextcloudappstore.core.models import NextcloudRelease
class GitHubClient:
def __init__(self, base_url: str, api_token: str = None) -> None:
self.base_url = base_url.rstrip('/')
self.api_token = api_token
self.headers = None if self.api_token else {
'Authorization': 'token %s' % self.api_token
}
def get_tags(self, page: int, size: int = 100):
url = '%s/repos/nextcloud/server/tags' % self.base_url
params = (('per_page', size), ('page', page))
response = requests.get(url, params=params, headers=self.headers)
response.raise_for_status()
return response.json()
def sync_releases(versions: Iterable[str]) -> None:
"""
All given versions have a release. If a release is absent, persisted
releases are out of date and need to have their release flag removed.
Finally the latest version must be marked as current.
:param versions: an iterable yielding all retrieved GitHub tags
:return:
"""
current_releases = NextcloudRelease.objects.all()
imported_releases = [
NextcloudRelease.objects.get_or_create(version=version)[0]
for version in versions
]
if imported_releases:
# all imported releases have a release, existing ones don't
for release in imported_releases:
release.has_release = True
release.save()
for release in get_old_releases(current_releases, imported_releases):
release.has_release = False
release.save()
# set latest release
NextcloudRelease.objects.update(is_current=False)
latest = max(imported_releases, key=lambda v: Version(v.version))
latest.is_current = True
latest.save()
NextcloudReleases = List[NextcloudRelease]
def get_old_releases(current: NextcloudReleases,
imported: NextcloudReleases) -> NextcloudReleases:
imported_versions = {release.version for release in imported}
return [release for release in current
if release.version not in imported_versions]
def get_supported_releases(client: GitHubClient,
oldest_supported: str) -> Iterable[str]:
releases = get_stable_releases(client)
return takewhile(lambda v: is_supported(v, oldest_supported), releases)
def get_stable_releases(client: GitHubClient) -> Iterable[str]:
json = chain.from_iterable(TagPages(client))
return (tag for tag in (release['name'].lstrip('v')
for release in json
if 'name' in release)
if is_stable(tag))
def is_supported(oldest_supported: str, version: str) -> bool:
return Version(oldest_supported) >= Version(version)
def is_stable(release: str) -> bool:
try:
version = Version(release)
return not version.prerelease
except ValueError:
return False
class TagPages:
"""
The GitHub API is paginated which makes it a pain to fetch all results.
This iterable returns a stream of json arrays until no further results
are found. To iterate over all releases you need to flatten the results
returned from this iterator first
"""
def __init__(self, client: GitHubClient, size: int = 100) -> None:
self.client = client
self.size = size
self.page = 1 # pages are 1 indexed
def __iter__(self) -> 'TagPages':
return self
def __next__(self):
json = self.client.get_tags(self.page, self.size)
if len(json) > 0:
self.page += 1
return json
else:
raise StopIteration

Просмотреть файл

@ -0,0 +1,36 @@
import requests
from django.conf import settings
from django.core.management import BaseCommand
from django.core.management import CommandError
from nextcloudappstore.core.github import get_supported_releases, \
GitHubClient, sync_releases
class Command(BaseCommand):
help = 'Queries Nextcloud\'s GitHub releases API to update all locally ' \
'stored Nextcloud versions'
def add_arguments(self, parser):
parser.add_argument('--print', required=False, action='store_true',
help='Prints to stdout instead of importing '
'releases into the database')
parser.add_argument('--oldest-supported', required=True,
help='Oldest supported Nextcloud version')
def handle(self, *args, **options):
oldest_supported = options.get('oldest_supported')
token = settings.GITHUB_API_TOKEN
base_url = settings.GITHUB_API_BASE_URL
client = GitHubClient(base_url, token)
try:
releases = get_supported_releases(client, oldest_supported)
except requests.HTTPError as e:
raise CommandError('Could not get releases: ' + str(e))
if options['print']:
for release in releases:
self.stdout.write(release)
else:
sync_releases(releases)

Просмотреть файл

@ -0,0 +1,272 @@
[
{
"name": "v13.0.0beta4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0beta4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0beta4",
"commit": {
"sha": "62656dc5c2b39f911fa3260ac291f0301b03a7a3",
"url": "https://api.github.com/repos/nextcloud/server/commits/62656dc5c2b39f911fa3260ac291f0301b03a7a3"
}
},
{
"name": "v13.0.0beta3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0beta3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0beta3",
"commit": {
"sha": "66391d65a68d89f5351c1be7810cd4abd0e82989",
"url": "https://api.github.com/repos/nextcloud/server/commits/66391d65a68d89f5351c1be7810cd4abd0e82989"
}
},
{
"name": "v13.0.0beta2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0beta2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0beta2",
"commit": {
"sha": "14c83e415c2e1e748c335fcb48a5b979240ecb9d",
"url": "https://api.github.com/repos/nextcloud/server/commits/14c83e415c2e1e748c335fcb48a5b979240ecb9d"
}
},
{
"name": "v13.0.0beta1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0beta1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0beta1",
"commit": {
"sha": "d906e16617de5abc0d48b0b7cbabe54f10cbed0b",
"url": "https://api.github.com/repos/nextcloud/server/commits/d906e16617de5abc0d48b0b7cbabe54f10cbed0b"
}
},
{
"name": "v13.0.0RC4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0RC4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0RC4",
"commit": {
"sha": "88d5f6ec637fd7b3e540b0616e0283b747a79969",
"url": "https://api.github.com/repos/nextcloud/server/commits/88d5f6ec637fd7b3e540b0616e0283b747a79969"
}
},
{
"name": "v13.0.0RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0RC3",
"commit": {
"sha": "d82d1a8a810780d73b2d5fa214ef110156b8285b",
"url": "https://api.github.com/repos/nextcloud/server/commits/d82d1a8a810780d73b2d5fa214ef110156b8285b"
}
},
{
"name": "v13.0.0RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0RC2",
"commit": {
"sha": "6719c7723a9b62e3dc5a60bf6f31068742d96880",
"url": "https://api.github.com/repos/nextcloud/server/commits/6719c7723a9b62e3dc5a60bf6f31068742d96880"
}
},
{
"name": "v13.0.0RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v13.0.0RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v13.0.0RC1",
"commit": {
"sha": "4f30c2fbe43e0d6f24ba3bff881526b2b6c2747d",
"url": "https://api.github.com/repos/nextcloud/server/commits/4f30c2fbe43e0d6f24ba3bff881526b2b6c2747d"
}
},
{
"name": "v12.0.5",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.5",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.5",
"commit": {
"sha": "42633b4d024fdad5f79c045e52afcedfc6d80f3e",
"url": "https://api.github.com/repos/nextcloud/server/commits/42633b4d024fdad5f79c045e52afcedfc6d80f3e"
}
},
{
"name": "v12.0.5RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.5RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.5RC3",
"commit": {
"sha": "74ee895cc7e3719190bf0780eee913e2dc82ebfb",
"url": "https://api.github.com/repos/nextcloud/server/commits/74ee895cc7e3719190bf0780eee913e2dc82ebfb"
}
},
{
"name": "v12.0.5RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.5RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.5RC2",
"commit": {
"sha": "533e59f1d5411d6ef502ea7ca551e7e6c9facd67",
"url": "https://api.github.com/repos/nextcloud/server/commits/533e59f1d5411d6ef502ea7ca551e7e6c9facd67"
}
},
{
"name": "v12.0.5RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.5RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.5RC1",
"commit": {
"sha": "323dc4acb12fced69cfc5f880d3e5efe830e4868",
"url": "https://api.github.com/repos/nextcloud/server/commits/323dc4acb12fced69cfc5f880d3e5efe830e4868"
}
},
{
"name": "v12.0.4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.4",
"commit": {
"sha": "c772810ecabce06f45dea16b7c67185c90df4a0d",
"url": "https://api.github.com/repos/nextcloud/server/commits/c772810ecabce06f45dea16b7c67185c90df4a0d"
}
},
{
"name": "v12.0.4RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.4RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.4RC3",
"commit": {
"sha": "aeae9aefe9dc361c40366775c74634b42903219c",
"url": "https://api.github.com/repos/nextcloud/server/commits/aeae9aefe9dc361c40366775c74634b42903219c"
}
},
{
"name": "v12.0.4RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.4RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.4RC2",
"commit": {
"sha": "ee69dc128a856f14419cb20b3cced4baf495d7cb",
"url": "https://api.github.com/repos/nextcloud/server/commits/ee69dc128a856f14419cb20b3cced4baf495d7cb"
}
},
{
"name": "v12.0.4RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.4RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.4RC1",
"commit": {
"sha": "b8d3577b92094bfcefd4cd740c1e3895e852dc26",
"url": "https://api.github.com/repos/nextcloud/server/commits/b8d3577b92094bfcefd4cd740c1e3895e852dc26"
}
},
{
"name": "v12.0.3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.3",
"commit": {
"sha": "343a69569db0aec48bc99fd3b23da08a7791d82b",
"url": "https://api.github.com/repos/nextcloud/server/commits/343a69569db0aec48bc99fd3b23da08a7791d82b"
}
},
{
"name": "v12.0.3RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.3RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.3RC2",
"commit": {
"sha": "51c110dd66ee3a0e56bb8179aaff2b95f6bb1319",
"url": "https://api.github.com/repos/nextcloud/server/commits/51c110dd66ee3a0e56bb8179aaff2b95f6bb1319"
}
},
{
"name": "v12.0.3RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.3RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.3RC1",
"commit": {
"sha": "499699f37b1b5acc9fbfc269804160ad87934d99",
"url": "https://api.github.com/repos/nextcloud/server/commits/499699f37b1b5acc9fbfc269804160ad87934d99"
}
},
{
"name": "v12.0.2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.2",
"commit": {
"sha": "3ecee6626bfea84f304bd0a6e041e706c3927f88",
"url": "https://api.github.com/repos/nextcloud/server/commits/3ecee6626bfea84f304bd0a6e041e706c3927f88"
}
},
{
"name": "v12.0.1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1",
"commit": {
"sha": "bfca9fbf115741e47a847ddb0116de1f82e8ba8d",
"url": "https://api.github.com/repos/nextcloud/server/commits/bfca9fbf115741e47a847ddb0116de1f82e8ba8d"
}
},
{
"name": "v12.0.1RC5",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1RC5",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1RC5",
"commit": {
"sha": "b500f8fbe9cdbd7cecb7ec951bdc0fb50d3687d4",
"url": "https://api.github.com/repos/nextcloud/server/commits/b500f8fbe9cdbd7cecb7ec951bdc0fb50d3687d4"
}
},
{
"name": "v12.0.1RC4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1RC4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1RC4",
"commit": {
"sha": "0a369747a0856035369fb072e7215fa92322ad17",
"url": "https://api.github.com/repos/nextcloud/server/commits/0a369747a0856035369fb072e7215fa92322ad17"
}
},
{
"name": "v12.0.1RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1RC3",
"commit": {
"sha": "2476a5f0e3e707b4dc23626b266b3ac8248529a1",
"url": "https://api.github.com/repos/nextcloud/server/commits/2476a5f0e3e707b4dc23626b266b3ac8248529a1"
}
},
{
"name": "v12.0.1RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1RC2",
"commit": {
"sha": "7c0e1c48601b6265f66da6d00878cdc952c75e99",
"url": "https://api.github.com/repos/nextcloud/server/commits/7c0e1c48601b6265f66da6d00878cdc952c75e99"
}
},
{
"name": "v12.0.1RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.1RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.1RC1",
"commit": {
"sha": "041e327a6c25ffdb35309e84fc7e9568ce98a414",
"url": "https://api.github.com/repos/nextcloud/server/commits/041e327a6c25ffdb35309e84fc7e9568ce98a414"
}
},
{
"name": "v12.0.0",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0",
"commit": {
"sha": "98e26f8b5c8b238e7f3556e900c524ce78bde95a",
"url": "https://api.github.com/repos/nextcloud/server/commits/98e26f8b5c8b238e7f3556e900c524ce78bde95a"
}
},
{
"name": "v12.0.0beta4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0beta4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0beta4",
"commit": {
"sha": "c002698ed2ce5cda07fbc8e88ce4f87d269826f8",
"url": "https://api.github.com/repos/nextcloud/server/commits/c002698ed2ce5cda07fbc8e88ce4f87d269826f8"
}
},
{
"name": "v12.0.0beta3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0beta3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0beta3",
"commit": {
"sha": "2c9007616bc4c985c9e101c5f9c306f42cf211c5",
"url": "https://api.github.com/repos/nextcloud/server/commits/2c9007616bc4c985c9e101c5f9c306f42cf211c5"
}
},
{
"name": "v12.0.0beta2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0beta2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0beta2",
"commit": {
"sha": "62f26101291b16429b0fcf00bf0a82bbd5655ab2",
"url": "https://api.github.com/repos/nextcloud/server/commits/62f26101291b16429b0fcf00bf0a82bbd5655ab2"
}
}
]

Просмотреть файл

@ -0,0 +1,272 @@
[
{
"name": "v12.0.0beta1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0beta1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0beta1",
"commit": {
"sha": "81ee0673a51ae2fe0a0b14420dce9e7337eb1425",
"url": "https://api.github.com/repos/nextcloud/server/commits/81ee0673a51ae2fe0a0b14420dce9e7337eb1425"
}
},
{
"name": "v12.0.0RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0RC3",
"commit": {
"sha": "e23569ed689c46ea2f5186591fabe8d31e2b6839",
"url": "https://api.github.com/repos/nextcloud/server/commits/e23569ed689c46ea2f5186591fabe8d31e2b6839"
}
},
{
"name": "v12.0.0RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0RC2",
"commit": {
"sha": "ebd15f50d9cad47d924cc9b27211350bbaa4fb0e",
"url": "https://api.github.com/repos/nextcloud/server/commits/ebd15f50d9cad47d924cc9b27211350bbaa4fb0e"
}
},
{
"name": "v12.0.0RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v12.0.0RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v12.0.0RC1",
"commit": {
"sha": "94b9335936e3f74ccf7418634b266c366c601cda",
"url": "https://api.github.com/repos/nextcloud/server/commits/94b9335936e3f74ccf7418634b266c366c601cda"
}
},
{
"name": "v11.0.7",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.7",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.7",
"commit": {
"sha": "b7e000da94043671ba3aea23d65e029f55299b83",
"url": "https://api.github.com/repos/nextcloud/server/commits/b7e000da94043671ba3aea23d65e029f55299b83"
}
},
{
"name": "v11.0.7RC3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.7RC3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.7RC3",
"commit": {
"sha": "f8f7fc9fc0a0b953dbaa596dc5d80e16f0cbd1ec",
"url": "https://api.github.com/repos/nextcloud/server/commits/f8f7fc9fc0a0b953dbaa596dc5d80e16f0cbd1ec"
}
},
{
"name": "v11.0.7RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.7RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.7RC2",
"commit": {
"sha": "a48a1ba670e83c58c9157610775b60c4065562fb",
"url": "https://api.github.com/repos/nextcloud/server/commits/a48a1ba670e83c58c9157610775b60c4065562fb"
}
},
{
"name": "v11.0.7RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.7RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.7RC1",
"commit": {
"sha": "406001ca848d7b9963a310599e4cdca0cd72cedb",
"url": "https://api.github.com/repos/nextcloud/server/commits/406001ca848d7b9963a310599e4cdca0cd72cedb"
}
},
{
"name": "v11.0.6",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.6",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.6",
"commit": {
"sha": "5dd382b21e689d390041f28101122632f9b851e6",
"url": "https://api.github.com/repos/nextcloud/server/commits/5dd382b21e689d390041f28101122632f9b851e6"
}
},
{
"name": "v11.0.6RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.6RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.6RC1",
"commit": {
"sha": "62e6e0f433d8a63d6bbd813550d93abd38d5df01",
"url": "https://api.github.com/repos/nextcloud/server/commits/62e6e0f433d8a63d6bbd813550d93abd38d5df01"
}
},
{
"name": "v11.0.5",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.5",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.5",
"commit": {
"sha": "f637a1a2281c47a46912e5ecf3d7418ea4a682bb",
"url": "https://api.github.com/repos/nextcloud/server/commits/f637a1a2281c47a46912e5ecf3d7418ea4a682bb"
}
},
{
"name": "v11.0.5RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.5RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.5RC1",
"commit": {
"sha": "5cb9790669be1d61f3fb2b5342298a45f1f9c129",
"url": "https://api.github.com/repos/nextcloud/server/commits/5cb9790669be1d61f3fb2b5342298a45f1f9c129"
}
},
{
"name": "v11.0.4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.4",
"commit": {
"sha": "9779e5eaceaf116c3b91af836923db352cfc0c0f",
"url": "https://api.github.com/repos/nextcloud/server/commits/9779e5eaceaf116c3b91af836923db352cfc0c0f"
}
},
{
"name": "v11.0.4RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.4RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.4RC1",
"commit": {
"sha": "40abc3a18a220f6b988e6609c6b62209e4a9a247",
"url": "https://api.github.com/repos/nextcloud/server/commits/40abc3a18a220f6b988e6609c6b62209e4a9a247"
}
},
{
"name": "v11.0.3",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.3",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.3",
"commit": {
"sha": "9649661f8fc0220b837975fe8afef5b5f7a964ec",
"url": "https://api.github.com/repos/nextcloud/server/commits/9649661f8fc0220b837975fe8afef5b5f7a964ec"
}
},
{
"name": "v11.0.3RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.3RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.3RC2",
"commit": {
"sha": "84383bc31645b2abc16f9a8975dac8dc92f794a1",
"url": "https://api.github.com/repos/nextcloud/server/commits/84383bc31645b2abc16f9a8975dac8dc92f794a1"
}
},
{
"name": "v11.0.3RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.3RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.3RC1",
"commit": {
"sha": "7ce627c308fc34e38a125847881d917f09dc54aa",
"url": "https://api.github.com/repos/nextcloud/server/commits/7ce627c308fc34e38a125847881d917f09dc54aa"
}
},
{
"name": "v11.0.2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.2",
"commit": {
"sha": "f299c9b2f4cb98b6dcd79943882740f9b72dc3c0",
"url": "https://api.github.com/repos/nextcloud/server/commits/f299c9b2f4cb98b6dcd79943882740f9b72dc3c0"
}
},
{
"name": "v11.0.2RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.2RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.2RC1",
"commit": {
"sha": "bd3c99057b4e2e82a984e66261ba5fe05ee707d7",
"url": "https://api.github.com/repos/nextcloud/server/commits/bd3c99057b4e2e82a984e66261ba5fe05ee707d7"
}
},
{
"name": "v11.0.1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.1",
"commit": {
"sha": "34e7a7225799cec35aa5e1b4eba68f573f9dbd98",
"url": "https://api.github.com/repos/nextcloud/server/commits/34e7a7225799cec35aa5e1b4eba68f573f9dbd98"
}
},
{
"name": "v11.0.1RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.1RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.1RC1",
"commit": {
"sha": "90c5912eec028b1019a899de15b270a6a18ea823",
"url": "https://api.github.com/repos/nextcloud/server/commits/90c5912eec028b1019a899de15b270a6a18ea823"
}
},
{
"name": "v11.0.0",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0.0",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0.0",
"commit": {
"sha": "ed7b18799f7d57ab9346e224b4dcea0961c9537f",
"url": "https://api.github.com/repos/nextcloud/server/commits/ed7b18799f7d57ab9346e224b4dcea0961c9537f"
}
},
{
"name": "v11.0RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v11.0RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v11.0RC2",
"commit": {
"sha": "36717f564e3d94767a8e3b9bd8a02bf695b94e93",
"url": "https://api.github.com/repos/nextcloud/server/commits/36717f564e3d94767a8e3b9bd8a02bf695b94e93"
}
},
{
"name": "v10.0.6",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.6",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.6",
"commit": {
"sha": "cf82aa3f3081ef3920bdcad9c848d36b1d11001b",
"url": "https://api.github.com/repos/nextcloud/server/commits/cf82aa3f3081ef3920bdcad9c848d36b1d11001b"
}
},
{
"name": "v10.0.6RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.6RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.6RC1",
"commit": {
"sha": "8b490e8a5c21997711ebcbfecbf1a3a1b336439d",
"url": "https://api.github.com/repos/nextcloud/server/commits/8b490e8a5c21997711ebcbfecbf1a3a1b336439d"
}
},
{
"name": "v10.0.5",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.5",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.5",
"commit": {
"sha": "0bd0635ee2424e0f8d967fee70fed16cf1c7bb06",
"url": "https://api.github.com/repos/nextcloud/server/commits/0bd0635ee2424e0f8d967fee70fed16cf1c7bb06"
}
},
{
"name": "v10.0.5RC2",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.5RC2",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.5RC2",
"commit": {
"sha": "d1b390a28f72ac8921bc6d8791bd12c0171f779a",
"url": "https://api.github.com/repos/nextcloud/server/commits/d1b390a28f72ac8921bc6d8791bd12c0171f779a"
}
},
{
"name": "v10.0.5RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.5RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.5RC1",
"commit": {
"sha": "ab0d4144d3e211dd36729618969fd11344656e8b",
"url": "https://api.github.com/repos/nextcloud/server/commits/ab0d4144d3e211dd36729618969fd11344656e8b"
}
},
{
"name": "v10.0.4",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.4",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.4",
"commit": {
"sha": "4520f350a0ed5c4d8b6218d2a636d3f7f8445aab",
"url": "https://api.github.com/repos/nextcloud/server/commits/4520f350a0ed5c4d8b6218d2a636d3f7f8445aab"
}
},
{
"name": "v10.0.4RC1",
"zipball_url": "https://api.github.com/repos/nextcloud/server/zipball/v10.0.4RC1",
"tarball_url": "https://api.github.com/repos/nextcloud/server/tarball/v10.0.4RC1",
"commit": {
"sha": "b3e42a6609eba53494616c304adf69339e0bd1d2",
"url": "https://api.github.com/repos/nextcloud/server/commits/b3e42a6609eba53494616c304adf69339e0bd1d2"
}
}
]

Просмотреть файл

@ -0,0 +1 @@
[]

Просмотреть файл

@ -0,0 +1,76 @@
import json
from typing import Dict, Any
from unittest.mock import MagicMock
from django.test import TestCase
from nextcloudappstore.core.facades import read_relative_file
from nextcloudappstore.core.github import GitHubClient, \
get_supported_releases, sync_releases
from nextcloudappstore.core.models import NextcloudRelease
class RatingTest(TestCase):
def test_parse_tags(self):
client = MagicMock(spec=GitHubClient)
client.get_tags.side_effect = self._get_tags
releases = get_supported_releases(client, '11.0.0')
expected = [
'11.0.0',
'11.0.1',
'11.0.2',
'11.0.3',
'11.0.4',
'11.0.5',
'11.0.6',
'11.0.7',
'12.0.0',
'12.0.1',
'12.0.2',
'12.0.3',
'12.0.4',
'12.0.5',
]
self.assertEquals(expected, sorted(list(releases)))
def test_import_releases(self):
client = MagicMock(spec=GitHubClient)
client.get_tags.side_effect = self._get_tags
releases = get_supported_releases(client, '11.0.0')
self._create_release('10.0.0', False, True)
self._create_release('10.0.1', True, True)
self._create_release('11.0.0', False, False)
self._create_release('11.0.1', True, False)
sync_releases(releases)
self.assertEquals(16, NextcloudRelease.objects.count())
self.assertEquals(False, self.get_rel('10.0.0').has_release)
self.assertEquals(False, self.get_rel('10.0.0').is_current)
self.assertEquals(False, self.get_rel('10.0.1').has_release)
self.assertEquals(False, self.get_rel('10.0.1').is_current)
self.assertEquals(True, self.get_rel('11.0.0').has_release)
self.assertEquals(False, self.get_rel('11.0.0').is_current)
self.assertEquals(True, self.get_rel('11.0.1').has_release)
self.assertEquals(False, self.get_rel('11.0.1').is_current)
self.assertEquals(True, self.get_rel('12.0.4').has_release)
self.assertEquals(False, self.get_rel('12.0.4').is_current)
self.assertEquals(True, self.get_rel('12.0.5').has_release)
self.assertEquals(True, self.get_rel('12.0.5').is_current)
def get_rel(self, version: str) -> NextcloudRelease:
return NextcloudRelease.objects.get(version=version)
def _create_release(self, version: str, is_current: bool,
has_release: bool) -> NextcloudRelease:
return NextcloudRelease.objects.create(version=version,
is_current=is_current,
has_release=has_release)
def _get_tags(self, page: int, size: int = 100) -> Dict[Any, Any]:
return json.loads(self._read('tags_page_%d.json' % page))
def _read(self, path: str) -> str:
return read_relative_file(__file__, 'data/%s' % path)

Просмотреть файл

@ -292,3 +292,7 @@ DISCOURSE_PARENT_CATEGORY_ID = 26
# list of values that will override or add user defined values for usage in
# templates; first key is the Nextcloud version for which the app is generated
APP_SCAFFOLDING_PROFILES = {}
# GitHub api configuration
GITHUB_API_BASE_URL = 'https://api.github.com'
GITHUB_API_TOKEN = None

Просмотреть файл

@ -7,7 +7,7 @@ chown nextcloudappstore:nextcloudappstore -R /srv/media
# adjust database schema and load new data into it
python manage.py migrate
python manage.py loaddata nextcloudappstore/**/fixtures/*.json
python manage.py loaddata nextcloudappstore/core/fixtures/*.json
python manage.py importdbtranslations
# copy data served by web server data into mounted volume

Просмотреть файл

@ -41,7 +41,7 @@ pip install -r requirements/production.txt
yarn install
yarn run build
python manage.py migrate
python manage.py loaddata nextcloudappstore/**/fixtures/*.json
python manage.py loaddata nextcloudappstore/core/fixtures/*.json
python manage.py collectstatic --noinput
python manage.py compilemessages
python manage.py importdbtranslations