зеркало из https://github.com/mozilla/FlightDeck.git
removed old commonware, docutils and sphinx
Cleaned commonware.log dependency Unsuccessful attempt to remove all django.db errors from ``make html`` command celery installation instructions added Info about vendor directory github location Some doc styling fixes * length of the underscore lines * code blocks Added RabbitMQ cheatsheet Less talking about Vincent Driessen's model Removed unnecessary branches description Removed code workflow diagram Less descriptive git CheatSheet for new developers Database migration doc added
This commit is contained in:
Piotr Zalewa
Родитель
e9901d6a57
Коммит
f632e5c741
|
|
@ -33,12 +33,6 @@ for item in list(sys.path):
|
|||
sys.path.remove(item)
|
||||
sys.path[:0] = new_sys_path
|
||||
|
||||
# XXX: for some reason this fixes the AttributeError exception
|
||||
#File "[...]/FlightDeck/apps/jetpack/managers.py", line 11, in <module>
|
||||
# log = commonware.log.getLogger('f.jetpack.managers')
|
||||
# AttributeError: 'module' object has no attribute 'log'
|
||||
import commonware.log
|
||||
|
||||
# Mock modules which fail on readthedocs.org
|
||||
class Mock(object):
|
||||
def __init__(self, *args):
|
||||
|
|
@ -47,7 +41,7 @@ class Mock(object):
|
|||
def __getattr__(self, name):
|
||||
return Mock
|
||||
|
||||
MOCK_MODULES = ['MySQLdb', 'waffle']
|
||||
MOCK_MODULES = ['MySQLdb', 'waffle', 'celery', 'celery.decorators', 'django.db.backends.mysql.base', 'django.shortcuts', 'django.db.models.signals']
|
||||
|
||||
for mod_name in MOCK_MODULES:
|
||||
sys.modules[mod_name] = Mock()
|
||||
|
|
|
|||
|
|
@ -4,74 +4,35 @@
|
|||
Code Workflow
|
||||
=============
|
||||
|
||||
Please read for a detailed explanation.
|
||||
|
||||
.. figure:: /_static/workflow.png
|
||||
:width: 250px
|
||||
|
||||
Code Workflow Diagram [:ref:`full size <code_workflow_diagram>`]
|
||||
We use a simplified `Vincent Driessen's model
|
||||
<http://nvie.com/git-model>`_
|
||||
|
||||
Decentralized, but Centralized
|
||||
==============================
|
||||
Every developer has his own repository called ``origin`` in this and following documents. There is only one referencing repository called ``main``. It will contain ``production``, ``staging``, ``release-*`` and ``master`` branches.
|
||||
|
||||
|
||||
Main branches
|
||||
=============
|
||||
|
||||
Main repository holds 3 infinite and a number of short term ``release-*`` branches.
|
||||
|
||||
production
|
||||
----------
|
||||
Must branch off from: ``staging``
|
||||
|
||||
This is the branch from which one can get a production ready code.
|
||||
|
||||
staging
|
||||
-------
|
||||
Must branch off from: ``release-*``
|
||||
Must branch back to: ``production``
|
||||
|
||||
This is the pranch from which a staging server can get the code.
|
||||
|
||||
release-*
|
||||
---------
|
||||
Must branch off from: ``master``
|
||||
Takes merges from: ``master`` or ``hotfix-*``
|
||||
Branches back to: ``staging``
|
||||
|
||||
This is the branch used to prepare the release. When we think it is ready, ``staging`` branch merges from it.
|
||||
|
||||
master
|
||||
------
|
||||
Takes merges from: ``bug-*``, ``hotfix-*``
|
||||
|
||||
This is our development branch - every developer's repository should have their ``origin/master`` in sync with the ``main/master``.
|
||||
|
||||
Origin branches
|
||||
===============
|
||||
|
||||
Origin is the remote repository owned by the developer
|
||||
|
||||
master
|
||||
------
|
||||
Updates from ``main/master``
|
||||
May merge back from ``bug-*`` and ``hotfix-*`` to provide local master repositories. It's usually not to be shared with other developers, especially ``main/master`` will not merge from it.
|
||||
Every developer has his own repository (we called ``origin`` in this and
|
||||
following documents). There is only one referencing *upstream* repository.
|
||||
``upstream/master`` branch is deployed to
|
||||
http://builder-addons-dev.allizom.org/ by Github's PUSH request.
|
||||
Production server is updated from tags (i.e. ``0.9.12``) by IT team.
|
||||
|
||||
bug-*
|
||||
-----
|
||||
Must branch of from ``master`` or other ``bug-*``.
|
||||
Naming convention: ``bug-{bugzilla_bug_id}-short_description``
|
||||
|
||||
Fixes one bug. It will be merged to the ``main`` repository by :ref:`Repository Manager <repository-manager>`
|
||||
Fixes one bug. It will be merged to the ``upstream.master``
|
||||
|
||||
production
|
||||
----------
|
||||
|
||||
If there is a serious bug in production we branch off tag to
|
||||
``upstream/production`` branch, fix it and tag it again.
|
||||
|
||||
hotfix-*
|
||||
--------
|
||||
Must branch off from ``main/release-*``, ``main/staging`` or ``main/production``
|
||||
Merges back to ``main/staging`` or ``main/production``
|
||||
Must branch off from ``upstream/production``
|
||||
Merges back to ``upstream/production`` and eventually
|
||||
``upstream/master``
|
||||
|
||||
This branch is for fixes to freezed code - in staging or production
|
||||
This branch is for fixes to freezed code - in production
|
||||
|
||||
.. References`
|
||||
|
||||
.. _`Vincent Driessen's model`: http://nvie.com/git-model
|
||||
|
|
|
|||
|
|
@ -1,6 +0,0 @@
|
|||
.. _code_workflow_diagram:
|
||||
|
||||
Code Workflow Diagram
|
||||
=====================
|
||||
|
||||
.. image:: /_static/workflow.png
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
|
||||
|
||||
Typical git commands used by developer
|
||||
======================================
|
||||
Cheatsheet - Typical git commands used by developer
|
||||
===================================================
|
||||
|
||||
Developer has a set of commands which are common for that workflow.
|
||||
Please take these advices as a starting point. They do not cover whole
|
||||
|
|
@ -13,7 +13,7 @@ Syncing master branch
|
|||
Master branch has to be usually in sync with the main master branch::
|
||||
|
||||
git checkout master
|
||||
git pull main master
|
||||
git pull upstream master
|
||||
|
||||
Fixing a bug
|
||||
------------
|
||||
|
|
@ -25,12 +25,6 @@ Create a branch with a bug number::
|
|||
|
||||
git checkout -b bug-12345-bug_description
|
||||
|
||||
.. note::
|
||||
``-b`` tells git to create a new branch. You may switch back to
|
||||
``master`` or other already created branch by::
|
||||
|
||||
git checkout branch_name`
|
||||
|
||||
.. note::
|
||||
If the bug is a hotfix it will be called hotfix-12345-branch_description
|
||||
|
||||
|
|
@ -39,11 +33,7 @@ Make some changes, publish the bug to the ``origin`` repository::
|
|||
git commit [ list_of_files | -a ] -v
|
||||
git push origin bug-12345-bug_description
|
||||
|
||||
.. note::
|
||||
``-v`` switch is to tell git to use ''verbose'' mode which opens an
|
||||
editor and displays a diff.
|
||||
|
||||
Send a pull request to the `Repository Manager <repository-manager>`.
|
||||
Send a pull request.
|
||||
|
||||
After the bug has been succesfully resolved the branch may be
|
||||
removed::
|
||||
|
|
@ -74,7 +64,3 @@ If you'd like later to load changes done by "fellow" - pull them from
|
|||
the remote branch::
|
||||
|
||||
git pull fellow/bug-12345-bug_description
|
||||
|
||||
.. note::
|
||||
Remember to checkout the ``bug-12345-bug_description`` before
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,33 @@
|
|||
.. _database_migration::
|
||||
|
||||
==================
|
||||
Database Migration
|
||||
==================
|
||||
|
||||
Add-ons Builder uses `Schematic <github.com/jbalogh/schematic.git>`_.
|
||||
*"The worst schema versioning system, ever?"*.
|
||||
|
||||
Usage
|
||||
#####
|
||||
|
||||
Applying migrations
|
||||
-------------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./vendor/src/schematic/schematic ./migrations/
|
||||
|
||||
|
||||
Creating migrations
|
||||
-------------------
|
||||
|
||||
Create ``migrations/{number}-{some_name}.[py/sql]`` file (check
|
||||
``migrations`` directory for examples). Python files will be executed and
|
||||
SQL run directly on database.
|
||||
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
|
||||
Schematic is storing current migration number in ``schema_version``
|
||||
table. Change it if you've created database by ``./manage.py syncdb``.
|
||||
|
|
@ -7,7 +7,13 @@ Vendor
|
|||
Vendor is a submodule. It contains all 3rd party libraries needed to run
|
||||
the FlightDeck on the server (i.e. Django).
|
||||
|
||||
There is a nice tool called `vending machine <https://github.com/jbalogh/vending-machine#readme>`_::
|
||||
One can change it's content. To do so please clone it's repository
|
||||
located at https://github.com/mozilla/FlightDeck-lib into a separate
|
||||
project.
|
||||
|
||||
There is a nice tool called `vending machine
|
||||
<https://github.com/jbalogh/vending-machine#readme>`_ which helps with
|
||||
management::
|
||||
|
||||
pip install -e git://github.com/jbalogh/vending-machine#egg=vend
|
||||
|
||||
|
|
@ -29,7 +35,7 @@ From the help::
|
|||
|
||||
However, because ``vendor`` is a submodule, vending-machine should not
|
||||
be used in it's default behavior. Instead, FlightDeck-lib should be
|
||||
checked out to a separate folder, and you should set the ``-d`` argument
|
||||
of vend::
|
||||
checked out from `it's repository https://github.com/mozilla/FlightDeck-lib`_
|
||||
to a separate folder, and you should set the ``-d`` argument of vend::
|
||||
|
||||
vend -d ./FlightDeck-lib add elasticutils
|
||||
|
|
|
|||
|
|
@ -121,7 +121,7 @@ and password you entered while running ``./manage.py syncdb``.
|
|||
You're all done!
|
||||
|
||||
Building documentation
|
||||
------------
|
||||
----------------------
|
||||
|
||||
FlightDeck uses `Sphinx <http://sphinx.pocoo.org/contents.html>`_-based documentation,
|
||||
so you have to install sphinx in order to build the docs::
|
||||
|
|
@ -143,7 +143,9 @@ Using Apache
|
|||
Production environments will expect to be running through another webserver.
|
||||
Some example Apache configurations follow.
|
||||
|
||||
An example Apache .conf::
|
||||
An example Apache .conf
|
||||
|
||||
..code-block::
|
||||
|
||||
<VirtualHost *:80>
|
||||
ServerAdmin your@mail.com
|
||||
|
|
@ -190,7 +192,10 @@ An example Apache .conf::
|
|||
WSGIScriptAlias / /path/to/FlightDeck/apache/config_local.wsgi
|
||||
</VirtualHost>
|
||||
|
||||
An example Apache WSGI configuration::
|
||||
|
||||
An example Apache WSGI configuration
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
|
@ -201,9 +206,9 @@ An example Apache WSGI configuration::
|
|||
|
||||
# All directories which should on the PYTHONPATH
|
||||
ALLDIRS = [
|
||||
os.path.join(VIRTUAL_ENV, 'lib/python2.6/site-packages'),
|
||||
PROJECT_PATH,
|
||||
os.path.join(PROJECT_PATH, 'flightdeck'),
|
||||
os.path.join(VIRTUAL_ENV, 'lib/python2.6/site-packages'),
|
||||
PROJECT_PATH,
|
||||
os.path.join(PROJECT_PATH, 'flightdeck'),
|
||||
]
|
||||
|
||||
# Remember original sys.path.
|
||||
|
|
@ -220,13 +225,13 @@ An example Apache WSGI configuration::
|
|||
# sys.path.append(workspace)
|
||||
|
||||
for s in ALLDIRS:
|
||||
sys.path.append(s)
|
||||
sys.path.append(s)
|
||||
|
||||
# reorder sys.path so new directories from the addsitedir show up first
|
||||
new_sys_path = [p for p in sys.path if p not in prev_sys_path]
|
||||
for item in new_sys_path:
|
||||
sys.path.remove(item)
|
||||
sys.path[:0] = new_sys_path
|
||||
sys.path.remove(item)
|
||||
sys.path[:0] = new_sys_path
|
||||
|
||||
os.environ['VIRTUAL_ENV'] = VIRTUAL_ENV
|
||||
os.environ['CUDDLEFISH_ROOT'] = VIRTUAL_ENV
|
||||
|
|
@ -238,24 +243,24 @@ An example Apache WSGI configuration::
|
|||
|
||||
|
||||
Recipes
|
||||
===============
|
||||
=======
|
||||
|
||||
|
||||
Import live database dump
|
||||
-------------------------
|
||||
|
||||
How to import a database dump from live
|
||||
How to import a database dump from live::
|
||||
|
||||
[sudo] mysql flightdeck < flightdeck_dump.sql
|
||||
|
||||
If you run into an error when importing large sql dump files, you may need to
|
||||
restart your mysqld process with this parameter.
|
||||
restart your mysqld process with this parameter::
|
||||
|
||||
mysqld --max_allowed_packet=32M
|
||||
|
||||
The database dump might be missing a row in django_sites table, so if you get a
|
||||
django error saying "Site matching query does not exist" when you hit the login
|
||||
page then insert a row into django_site.
|
||||
page then insert a row into django_site::
|
||||
|
||||
insert into django_site (id,domain,name) values (1,'example.com','example')
|
||||
|
||||
|
|
@ -265,7 +270,7 @@ After importing the data, you will need to rebuild your ES index.
|
|||
Rebuilding Elastic Search index
|
||||
-------------------------------
|
||||
|
||||
Need to delete your Elastic Search index and start over?
|
||||
Need to delete your Elastic Search index and start over?::
|
||||
|
||||
curl -XDELETE 'http://localhost:9201/flightdeck'
|
||||
./manage.py cron setup_mapping
|
||||
|
|
@ -275,8 +280,50 @@ Need to delete your Elastic Search index and start over?
|
|||
Create a local super user account
|
||||
---------------------------------
|
||||
|
||||
If you imported your database then you will need to create a user.
|
||||
If you imported your database then you will need to create a user::
|
||||
|
||||
./manage.py createsuperuser
|
||||
|
||||
|
||||
Using with Celery
|
||||
-----------------
|
||||
|
||||
Majority of resources heavy tasks is done by delegating them to celery.
|
||||
|
||||
By default on development boxes celery is not running and tasks are run
|
||||
synchronously. To be able to test celery tasks one has to configure the
|
||||
development system to resemble the production one.
|
||||
|
||||
Celery requires a running messaging system. We use
|
||||
`RabbitMQ <http://www.rabbitmq.com/>`_.
|
||||
|
||||
To configure please copy the Celery section from ``settings.py`` to
|
||||
``settings_local.py`` and uncomment it.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# These settings are for if you have celeryd running
|
||||
BROKER_HOST = 'localhost'
|
||||
BROKER_PORT = 5672
|
||||
BROKER_USER = 'builder'
|
||||
BROKER_PASSWORD = 'builder'
|
||||
BROKER_VHOST = 'builder'
|
||||
BROKER_CONNECTION_TIMEOUT = 0.1
|
||||
CELERY_RESULT_BACKEND = 'amqp'
|
||||
CELERY_IGNORE_RESULT = True
|
||||
|
||||
**RabbitMQ CheatSheet**
|
||||
|
||||
Create user, virtual host and give user all privileges::
|
||||
|
||||
sudo rabbitmqctl add_user builder builder
|
||||
sudo rabbitmqctl add_vhost builder
|
||||
sudo rabbitmqctl set_permission -p builder builder ".*" ".*" ".*"
|
||||
|
||||
|
||||
From project directory run::
|
||||
|
||||
./manage.py celeryd -l INFO
|
||||
|
||||
|
||||
.. _RabbitMQ: http://www.rabbitmq.com/
|
||||
|
|
|
|||
2
vendor
2
vendor
|
|
@ -1 +1 @@
|
|||
Subproject commit d1f254bb0038dd4e6a4f4ab31b737b1822bc0a9c
|
||||
Subproject commit 8c926b8144ea06233e2e62c3e9d9f762cfbddaf6
|
||||
Загрузка…
Ссылка в новой задаче