Removed playdoh docs from docs/ and made it a template for the forked projects. Playdoh docs now live under gh.com/mozilla/playdoh-docs

Closes issue 31.
This commit is contained in:
Fred Wenzel 2011-06-09 15:31:18 -07:00
Родитель c6d7e2abbb
Коммит c6a1c1b78b
11 изменённых файлов: 18 добавлений и 833 удалений

Просмотреть файл

@ -1,79 +0,0 @@
.. _bestpractices:
==============
Best Practices
==============
This page lists several best practices for writing secure web applications
with playdoh.
``|safe`` considered harmful
----------------------------
Using something like ``mystring|safe`` in a template will prevent Jinja2 from
auto-escaping it. Sadly, this requires us to be really sure that ``mystring``
is not raw, user-entered data. Otherwise we introduce an XSS vulnerability.
We have therefore eliminated the need for ``|safe`` for localized strings. This
works::
{{ _('Hello <strong>world</strong>!') }}
String interpolation
~~~~~~~~~~~~~~~~~~~~
When you *interpolate* data into such a string, however, the resulting output
will lose its "safeness" and be escaped again. To mark the *localized part*
of an interpolated string as safe, do not use ``|f(...)|safe``. The data could
be unsafe. Instead, use the helper ``|fe(...)``. It will escape all its
arguments before doing string interpolation, then return HTML that's safe to
use::
{{ _('Welcome back, <strong>{username}</strong>!')|fe(username=user.display_name) }}
``|f(...)|safe`` is to be considered unsafe and **should not pass code
review**.
If you interpolate into a base string that does *not contain HTML*, you may
keep on using ``|f(...)`` without ``|safe``, of course, as the auto-escaping
won't harm anything::
{{ _('Author name: {author}')|f(author=user.display_name) }}
Form fields
~~~~~~~~~~~
Jinja2, unlike Django templates, by default does not consider Django forms
"safe" to display. Thus, you'd use something like ``{{ form.myfield|safe }}``.
In order to minimize the use of ``|safe`` (and thus possible unsafe uses of
it), playdoh monkey-patches the Django forms framework so that form fields'
HTML representations are considered safe by Jinja2 as well. Therefore, the
following works as expected::
{{ form.myfield }}
Mmmmh, Cookies
--------------
Django's default way of setting a cookie is set_cookie_ on the HTTP response.
Unfortunately, both **secure** cookies (i.e., HTTPS-only) and **httponly**
(i.e., cookies not readable by JavaScript, if the browser supports it) are
disabled by default.
To be secure by default, we use commonware's ``cookies`` app. It makes secure
and httponly cookies the default, unless specifically requested otherwise.
To disable either of these patches, set ``COOKIES_SECURE = False`` or
``COOKIES_HTTPONLY = False`` in ``settings.py``.
You can exempt any cookie by passing ``secure=False`` or ``httponly=False`` to
the ``set_cookie`` call, respectively::
response.set_cookie('hello', value='world', secure=False, httponly=False)
.. _set_cookie: http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpResponse.set_cookie

Просмотреть файл

@ -1,5 +1,8 @@
#!/bin/zsh #!/bin/zsh
# A useful build script for projects hosted on github:
# It can build your Sphinx docs and push them straight to your gh-pages branch.
# Should be run from the docs directory: (cd docs && ./build-github.zsh) # Should be run from the docs directory: (cd docs && ./build-github.zsh)
REPO=$(git config remote.origin.url) REPO=$(git config remote.origin.url)

Просмотреть файл

@ -40,8 +40,8 @@ source_suffix = '.rst'
master_doc = 'index' master_doc = 'index'
# General information about the project. # General information about the project.
project = u'playdoh' project = u'a playdoh-based project'
copyright = u'2011, Mozilla' copyright = u'2011, the authors'
# The version info for the project you're documenting, acts as replacement for # The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the # |version| and |release|, also used in various other places throughout the
@ -211,8 +211,8 @@ latex_documents = [
# One entry per manual page. List of tuples # One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section). # (source start file, name, description, authors, manual section).
man_pages = [ man_pages = [
('index', 'playdoh', u'playdoh Documentation', ('index', 'a-playdoh-app', u"a-playdoh-app's Documentation",
[u'Mozilla'], 1) [u'the authors'], 1)
] ]

Просмотреть файл

@ -1,81 +0,0 @@
.. _docs:
=========================
Documentation with Sphinx
=========================
*(Borrowed from Zamboni)*
For a reStructuredText Primer, see http://sphinx.pocoo.org/rest.html.
Sections
--------
Sphinx doesn't care what punctuation you use for marking headings, but each new
kind that it sees means one level lower in the outline. So, ::
=========
Heading 1
=========
Heading 2
---------
will correspond to ::
<h1>Heading 1</h1>
<h2>Heading 2</h2>
For consistency, this is what I'm using for headings in Zamboni::
=========
Heading 1
=========
Heading 2
---------
Heading 3
~~~~~~~~~
Heading 4
*********
Heading 5
+++++++++
Heading 6
^^^^^^^^^
Use two newlines prior to a new heading. I'm only using one here for space
efficiency.
Sphinx Extras
-------------
Use ``:src:`path/to/file.py``` to link to source files online. Example:
:src:`settings.py`.
Vim
---
Here's a nice macro for creating headings::
let @h = "yypVr"
Compiling Documentation
-----------------------
Playdoh hosts its documentation on `github pages
<http://mozilla.github.com/playdoh/>`_. When you change the docs, make sure
they still build properly and look all right locally::
cd docs && make html && open _build/html/index.html
If they do, run a build and push it to gh-pages::
./build-github.zsh

Просмотреть файл

@ -1,46 +0,0 @@
Getting started
===============
Requirements
------------
You need Python 2.6.
To check out playdoh, run::
git clone --recursive git://github.com/mozilla/playdoh.git
This project is set up to use a vendor library, i.e. a subdirectory ``vendor``
that contains all pure Python libraries required by this project. The recursive
checkout will also clone these requirements.
In addition, there are compiled libraries (such as Jinja2) that you will need
to build yourself, either by installing them from ``pypi`` or by using your
favorite package manager for your OS.
For development, you can run this in a `virtualenv environment`_::
easy_install pip
pip install -r requirements/compiled.txt
For more information on vendor libraries, read :ref:`packages`.
.. _virtualenv environment: http://pypi.python.org/pypi/virtualenv
Starting a project based on playdoh
-----------------------------------
The default branch of playdoh is ``base``. To start a new project, you fork
playdoh and start working on your app in ``master`` (branched from base). If
you start adding pieces that should go back into playdoh, you can apply the
patch to base and move it upstream.
Eventually you'll probably diverge enough that you'll want to delete the base
branch.
Publishing your repo
--------------------
git remote rename origin playdoh
git remote add origin git@github.com:mozilla/foobar.git
git push -u origin base

Просмотреть файл

@ -1,34 +1,19 @@
=================================== ========================================
Welcome to playdoh's documentation! Welcome to this project's documentation!
=================================== ========================================
**Mozilla's Playdoh** is a web application template based on Django_. This is a documentation template for a **web application based on Playdoh**.
Feel free to change this to your liking.
Patches are welcome! Feel free to fork and contribute to this project on
Github_.
.. _Django: http://www.djangoproject.com/
.. _Github: https://github.com/mozilla/playdoh
Features About playdoh
-------- -------------
Quick and dirty (and probably incomplete) feature list:
* Rich, but "cherry-pickable" features out of the box: This project is based on **playdoh**. Mozilla's Playdoh is an open source
web application template based on `Django <http://www.djangoproject.com/>`_.
* Django
* jinja2 template engine
* Celery support
* Simple database migrations
* Full localization support
* Secure by default:
* SHA-512 default password hashing
* X-Frame-Options: DENY by default
* secure and httponly flags on cookies enabled by default
To learn more about it, step by the `playdoh project page
<https://github.com/mozilla/playdoh>`_.
Contents Contents
-------- --------
@ -36,15 +21,6 @@ Contents
.. toctree:: .. toctree::
:maxdepth: 1 :maxdepth: 1
gettingstarted
libs
operations
migrations
l10n
packages
docs
bestpractices
Indices and tables Indices and tables
------------------ ------------------

Просмотреть файл

@ -1,135 +0,0 @@
Localization (L10n)
===================
So you'd like to localize your webapp. **Congratulations!**
Playdoh comes with all of the libraries and tools you need in the dev and
production requirements.
Requirements
------------
The one set of low level tools not provided is gettext.
::
aptitude install gettext
# or
brew install gettext
Environment Setup
-----------------
Your top level directory named locale should be an SVN repository. This is
because localizers are used to SVN and some tools like Pootle tie into it.
Developers who never deal with L10n do not need to worry about this kind of
setup. It's often easier to have a L10n dev setup, which a cron job updates
the sources, updates the strings, and compiles the po files and lastly
commiting to SVN.
Initial Steps
-------------
To allow localizers to translate your app on localize.mozilla.org, your .po
files need to be in SVN. You want a checkout of your locale directory to live
inside the git checkout under ``locale``.
The following steps will get you started:
#. Create an SVN repository with locale-based directories and ``templates``.
You need at least::
./en_US
./en_US/LC_MESSAGES
./templates
./templates/LC_MESSAGES
#. Check out this SVN repository inside your project path::
cd /my/playdoh/project/dir/
svn checkout https://svn.mozilla.org/projects/something/trunk/locale locale
## If you don't like svn, you may use git-svn too:
# git svn clone https://svn.mozilla.org/projects/something/trunk/locale locale
#. Now extract strings from your project::
./manage.py extract
./manage.py merge
#. Commit it all.
#. Build your .mo files and commit those as well::
./bin/compile-mo.sh locale/
cd locale
git add .
git commit -m 'Built .mo files'
Extracting Strings
------------------
If you've developed a feature and changed or added strings, you need to extract
them and merge them into the existing .po files::
./manage.py extract
./manage.py merge
Optionally compile all .mo files::
./bin/compile-mo.sh locale/
Creating New Locales
--------------------
Assuming you want to add 'fr':
#. ``mkdir -p locale/fr/LC_MESSAGES``
#. ``./manage.py merge``
or
#. ``msginit --no-translator -l fr -i templates/LC_MESSAGES/messages.pot -o fr/LC_MESSAGES/messages.po``
#. repeat for other POT files
Advanced Steps
--------------
#. Some projects keep a copy of the SVN checkout in git. When doing that,
keep in mind that ``git svn dcommit`` will change the commit message,
thus changing the git commit ID. Therefore, always commit to SVN first,
then push to git, never vice-versa::
cd locale/
# do something
git commit -am 'did something'
git svn dcommit && git push origin HEAD
#. localize.mozilla.org does not automatically compile .mo files when a
localizer commits to SVN. Neither does it automate the push over to
git. There's a handy little script called ``autol10n`` that you can
run as a cron job somewhere, which will:
* pick up changed .po files from SVN,
* compile the .mo files,
* then (if necessary) push everything over to git...
* ... and retag the git submodule to the latest revision.
The script is available in playdoh under ``bin/autol10n.sh``.
Q&A
---
* *Why SVN?* Our localizers like to use either SVN or Verbatim.
* *Why a git repo to mirror an SVN repo?* This allows us to have an external
reference ("git submodule") and deploy the app including its translations
easily.
* *How do I use gettext?* In templates we use jinja_
.. _jinja: http://jinja.pocoo.org/docs/templates/#i18n

Просмотреть файл

@ -1,115 +0,0 @@
=========
Libraries
=========
There are a number of libraries that either come bundled with playdoh or are
otherwise useful for it.
**Note:** Libraries marked with an \*asterisk\* are bundled with playdoh by default.
Python
======
Database
--------
* `django-multidb-router <https://github.com/jbalogh/django-multidb-router>`_:
Round-robin master/slave db router for Django 1.2.
* `schematic <https://github.com/jbalogh/schematic>`_\*:
Simple database migration tool.
Deferred Execution (cron, message queues)
-----------------------------------------
* `django-celery <https://github.com/ask/django-celery>`_\*:
Celery integration for Django.
* `django-cronjobs <https://github.com/jsocol/django-cronjobs>`_:
A simple cron-running management command for Django.
* `django-gearman <https://github.com/fwenzel/django-gearman>`_:
A convenience wrapper for Gearman clients and workers in Django/Python.
Deployment
----------
* `django-waffle <https://github.com/jsocol/django-waffle>`_:
A feature flipper for Django.
Internationalization (i18n) and Localization (L10n)
---------------------------------------------------
* `Babel <http://babel.edgewall.org/>`_\*:
A collection of tools for internationalizing Python applications.
* `pytz <http://pytz.sourceforge.net/>`_:
World Timezone Definitions for Python.
* `tower <https://github.com/clouserw/tower>`_\*:
A library that builds on Babel and Jinja2 to make extracting strings easy and
uniform.
* `The Translate toolkit <http://translate.sourceforge.net/wiki/toolkit/index>`_\*:
Tools for working between translation formats.
Middleware
----------
* `commonware <http://github.com/jsocol/commonware>`_\*:
Middlewares and other stuff we share.
* `django-mobility <https://github.com/jbalogh/django-mobility>`_:
Middleware and decorators for directing users to your mobile site.
Mozilla
-------
* `django-moz-header <https://github.com/mozilla/django-moz-header>`_:
Common header/footer templates and CSS for Django-based Mozilla sites.
* `django-mozilla-product-details <http://github.com/fwenzel/django-mozilla-product-details>`_:
Pulls Mozilla product details library data from SVN and makes it available
as Python dictionaries.
Security and Data Sanitization
------------------------------
* `bleach <https://github.com/jsocol/bleach>`_:
An easy, HTML5, whitelisting HTML sanitizer.
* `django-sha2 <http://github.com/fwenzel/django-sha2>`_\*:
Monkey-patches strong password hashing support into Django.
* `happyforms <https://github.com/jbalogh/happyforms>`_:
Extension to Django Forms that strips spaces.
Templates and Caching
---------------------
* `django-cache-machine <https://github.com/jbalogh/django-cache-machine>`_:
Automatic caching and invalidation for Django models through the ORM.
* `jingo <https://github.com/jbalogh/jingo>`_\*:
An adapter for using Jinja2 templates with Django.
* `jingo-minify <https://github.com/jsocol/jingo-minify>`_\*:
Concatenate and minify JS and CSS for Jinja2 + Jingo + Django.
* `hera <https://github.com/clouserw/hera>`_:
Client for Zeus Traffic Manager SOAP API.
Testing
-------
* `django-fixture-magic <https://github.com/davedash/django-fixture-magic>`_:
Utilities to extract and manipulate Django fixtures.
* `django-nose <https://github.com/jbalogh/django-nose>`_\*:
Django test runner using *nose*.
* `nose <http://somethingaboutorange.com/mrl/projects/nose/>`_\*:
*nose* extends unittest to make testing easier.
* `test-utils <https://github.com/jbalogh/test-utils>`_\*:
A grab-bag of testing utilities that are pretty specific to our Django,
Jinja2, and nose setup.
Various
-------
* `nuggets <https://github.com/mozilla/nuggets/>`_\*:
Little utilities that don't deserve a package.
JavaScript
==========
* `jQuery <http://jquery.com/>`_\*:
The jQuery JavaScript library.

Просмотреть файл

@ -1,46 +0,0 @@
.. _migrations:
===================
Database Migrations
===================
For database migrations, we use a strikingly simple tool called `schematic
<https://github.com/jbalogh/schematic>`_.
It runs numbered SQL files on top of your existing database. In a table
named ``schema_version``, it remembers what migration number your database
is currently on. If the table does not exist, it considers the database
to be at migration 0 and creates the table.
By default, migrations live in the ``migrations/`` directory. Run schematic
like this::
schematic migrations/
# or, when using the vendor library:
./vendor/src/schematic/schematic migrations/
Schematic vs. syncdb
--------------------
To create a DB from scratch, Django users run ``./manage.py syncdb``. It
analyzes the models and generates SQL accordingly.
Unfortunately, as the schema evolves, the output of *syncdb* will conflict
with the *schematic* migrations you might have added to evolve older DBs.
To circumvent this, consider the following:
When done creating the first set of models, capture their syncdb-created SQL
output as a baseline migration (number 1)::
# edit some models in apps/foo/models.py
./manage.py sqlall foo > migrations/01-base.sql
Then, as the schema evolves, add incremental migrations as ``02-...sql``, etc.
When setting up a copy of the codebase from scratch in the future, forgo
*syncdb* altogether and after setting your empty database's credentials
correctly in ``settings_local.py``, just run ``schematic migrations/``. It will
run your base migration, followed by incremental updates, and result in a
database that's up to date with your latest existing databases.

Просмотреть файл

@ -1,100 +0,0 @@
Operations
==========
Care and feeding of a Playdoh-based app.
Web Server
----------
Apps are typically run under Apache and mod_wsgi in production. Entry point::
wsgi/playdoh.wsgi
(or whatever you rename it to...)
Developers can set that up or run in stand-alone mode::
./manage.py runserver 0.0.0.0:8000
It is critical that developers run the app at least once via mod_wsgi before
certifying an app as **stage ready**.
Apache
~~~~~~
This is a typical virtualhost directive being used in production::
<VirtualHost *:80>
ServerName %HOSTNAME%
Alias /media %APP_PATH/media
WSGIScriptAlias / %APP_PATH/wsgi/playdoh.wsgi
WSGIDaemonProcess playdoh processes=16 threads=1 display-name=playdoh
WSGIProcessGroup playdoh
</VirtualHost>
gunicorn
~~~~~~~~
Totally optional and only for the cool kids.
A lighter weight method of testing your mod_wsgi setup is by using
`gunicorn <http://gunicorn.org/>`_.
One time setup::
pip install gunicorn
ln -s wsgi/playdoh.wsgi wsgi/playdoh.py
Each Time::
touch wsgi/__init__.py
gunicorn wsgi/playdoh:application
Middleware Caching
------------------
TODO (memcache, redis)
Frontend Caching
----------------
Apps are typically run behind a Zeus load balancer.
Database
--------
Apps typically use a MySQL database.
Message Queue
-------------
Playdoh comes packaged with celery and works well with RabbitMQ.
Updating your Environment
-------------------------
You can run ``update_site.py`` to keep your app current.
It does the following:
* Updates source
* Updates vendor libraries
* Runs Database Migrations
* Builds JS and CSS
::
./bin/update_site.py -e dev
./bin/update_site.py -e stage
./bin/update_site.py -e prod
You may pass a ``-v`` and update_site will explain what commands as it runs
them.
If there is an error on any step, the script stops.
IT will typically put ``bin/update_site.py`` into a cron for auto-deployment
to stage environments.
Edit your copy to customize your branching and/or release practices.

Просмотреть файл

@ -1,192 +0,0 @@
.. _packages:
==========================
pip and friends: Packaging
==========================
*(largely borrowed from Zamboni)*
Your app will depend on lots of tasty open source Python libararies. The list
of all your dependencies should exist in two places:
# requirements/prod.txt
# As a submodule of vendor
Ultimately your app code will run against the libraries under vendor via mod_wsgi.
Why requirements? For developement, you can use virtualenvs and pip to have a
tidy self-contained environment. If run from the Django runserver command, you
don't even need a web server.
The vendor library
------------------
The ``/vendor`` library is supposed to contain all packages and repositories.
It enables the project to be deployed as one package onto many machines,
without relying on PyPI-based installations on each target machine.
By default, the vendor lib is checked out as a *git submodule* under
``vendor/``. If you *do* need to check it out separately, do::
git clone --recursive git://github.com/mozilla/playdoh-lib.git ./vendor
Once the playdoh-lib repo has been downloaded to ``/vendor``, you only need to
install the **compiled** packages (as defined in ``requirements/compiled.txt``).
These can come from your system package manager or from::
pip install -r requirements/compiled.txt
Global vs. local library
------------------------
Playdoh provides its default library in the ``vendor/`` directory. You *may*
fork and change it, but that will make it hard to pull updates from the
upstream library later.
If you want to make only a few **local additions** or override some of the
libs in ``vendor/``, make those changes to the directory ``vendor-local/``
instead, which (in ``manage.py``) is given precedence over playdoh's vendor
dir.
compiled.txt vs prod.txt
------------------------
If a Python library requires compilation, it should be recorded in compiled.txt.
These aren't as portable and cannot be shipped in the vendor library.
For local development, it's nice to pip install these into a virtualenv. A
common practise is to use virtual env **only** for compiled libraries and
vendor for the rest of your dependencies.
Adding new packages
-------------------
If we wanted to add a new dependency called ``cheeseballs`` to playdoh, you
would add it to ``requirements/prod.txt``. If your library isn't used in
production, then put it in ``requirements/dev.txt``. This makes it available
to users installing into virtualenvs.
We also need to add the new package to the vendor lib, since that is what runs
in production...
First, we need to add the source. There are two ways, depending on how
this project is hosted:
Non-git based repos (hg, CVS, tarball)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For such repos or for packages coming from PyPI, do::
pip install -I --install-option="--home=`pwd`/vendor-local" cheeseballs
cd vendor-local
git add lib/python/cheeseballs
git commit
Optionally, if the package installs executables add them too. For
example::
cd vendor-local
git add bin/cheeseballer
git commit
For hg repos that are not on PyPI, they can be installed with pip too
but omit the ``--home`` option and use the ``--src`` instead. For
example::
pip install -I --src='vendor-local/src' \
-e hg+http://bitbucket.org/jespern/django-piston@default#egg=django-piston
cd vendor-local
git add src/django-piston
git commit
Note: Installed source packages need to be appended to
``vendor-local/vendor.pth``. See note below. For example::
echo src/django-piston >> vendor-local/vendor.pth
git-based repositories
~~~~~~~~~~~~~~~~~~~~~~
For a git-based package, add it as a git submodule::
cd vendor-local
git submodule add git://github.com/mozilla/cheeseballs.git src/cheeseballs
git commit vendor.pth .gitmodules src/cheeseballs
Further, you then need to update ``vendor-local/vendor.pth``. Python uses
``.pth`` files to dynamically add directories to ``sys.path`` (`docs
<http://docs.python.org/library/site.html>`_).
The file format is simple. Consult ``vendor/vendor.pth`` for reference.
Some packages (like ``html5lib`` and ``selenium``) are troublesome, because
their source lives inside an extra subdirectory ``src/`` inside their checkout.
So they need to be sourced with ``src/html5lib/src``, for example. Hopefully
you won't hit any snags like that.
Done. Try ``./manage.py shell`` and then ``import cheeseballs`` to make sure
it worked.
Testing Your Vendor Change
---------------
It's critical that you test your app running under mod_wsgi. Although you
may use runserver day to day, go ahead and run some code through WSGI to
prove vendor is setup properly. (throw an import into your view, etc)
Advanced Topics
---------------
TODO [automate these instructions](<https://github.com/mozilla/playdoh/issues/30)
Initial creation of the vendor library
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The vendor repo was seeded with ::
pip install -I --install-option="--home=`pwd`/vendor" --src='vendor/src' -r requirements/dev.txt
# ..delete some junk from vendor/lib/python...
# Create the .pth file so Python can find our src libs.
find src -type d -depth 1 >> vendor.pth
# Add all the submodules.
for f in src/*; do
pushd $f >/dev/null && REPO=$(git config remote.origin.url) && popd > /dev/null && git submodule add $REPO $f
done
git add .
Adding lots of git submodules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As noted in *Adding new packages*, git-based packages are *git submodules*
inside the vendor library. To set up the first batch of submodules, something
like the following happened::
for f in src/*
pushd $f && REPO=$(git config remote.origin.url) && popd && git submodule add $REPO $f
For reference: pip
~~~~~~~~~~~~~~~~~~
The classical method of installing is using pip. We have our packages
separated into three files:
:src:`requirements/compiled.txt`
All packages that require (or go faster with) compilation. These can't be
distributed cross-platform, so they need to be installed through your
system's package manager or pip.
:src:`requirements/prod.txt`
The minimal set of packages you need to run zamboni in production. You
also need to get ``requirements/compiled.txt``.
:src:`requirements/dev.txt`
All the packages needed for running tests and development servers. This
automatically includes ``requirements/prod.txt``.
With pip, you can get a development environment with::
pip install -r requirements/dev.txt -r requirements/compiled.txt