зеркало из https://github.com/mozilla/bedrock.git
1090 строки
41 KiB
ReStructuredText
1090 строки
41 KiB
ReStructuredText
.. This Source Code Form is subject to the terms of the Mozilla Public
|
|
.. License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
.. file, You can obtain one at https://mozilla.org/MPL/2.0/.
|
|
|
|
.. _l10n:
|
|
|
|
============
|
|
Localization
|
|
============
|
|
|
|
The site is fully localizable. Localization files are not shipped with the code
|
|
distribution, but are available in separate GitHub repositories. The proper repos
|
|
can be cloned and kept up-to-date using the ``l10n_update`` management command:
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_update
|
|
|
|
If you don't already have a ``data/locale`` directory, this command will clone the
|
|
git repo containing the .lang translation files (either the dev or prod files
|
|
depending on your ``DEV`` setting). If the folder is already present, it will
|
|
update the repository to the latest version. It does the same thing for the
|
|
repository for the .ftl translation files in ``data/www-l10n`` directory.
|
|
|
|
Fluent
|
|
======
|
|
|
|
Bedrock's Localization (l10n) system is based on `Project Fluent`_. This is a
|
|
departure from the legacy system (see below) that relied on a gettext work flow
|
|
of string extraction from template and code files, in that it relies on developers
|
|
directly editing the default language (English in our case) Fluent files and using
|
|
the string IDs created there in their templates and views.
|
|
|
|
The default files for the Fluent system live in the ``l10n`` directory in the root
|
|
of the bedrock project. This directory houses directories for each locale the developers
|
|
directly implement (mostly simplified English "en", and "en-US"). The simplified English
|
|
files are the default fallback for every string ID on the site and should be strings that
|
|
are plain and easy to understand English, as free from colloquialisms as possible. The
|
|
translators are able to easily understand the meaning of the string, and can then add their
|
|
own local flair to the ideas.
|
|
|
|
.. note::
|
|
|
|
We have some :ref:`fluent tools <fluent>` to aid in the transition from the legacy system.
|
|
|
|
.. _Project Fluent: https://projectfluent.org/
|
|
|
|
.ftl files
|
|
----------
|
|
|
|
When adding translatable strings to the site you start by putting them all into an .ftl
|
|
file in the ``l10n/en/`` directory with a path that matches or is somehow meaningful
|
|
for the expected location of the template or view in which they'll be used. For example,
|
|
strings for the ``mozorg/mission.html`` template would go into the ``l10n/en/mozorg/mission.ftl``
|
|
file. Locales are activated for a particular .ftl file, not template or URL, so you should use
|
|
a unique file for most URLs, unless they're meant to be translated and activated for new locales
|
|
simultaneously.
|
|
|
|
You can have shared .ftl files that you can load into any template render, but only the first
|
|
.ftl file in the list of the ones for a page render will determine whether the page is active
|
|
for a locale.
|
|
|
|
Activation of a locale happens automatically once certain rules are met. A developer can mark
|
|
some string IDs as being "Required", which means that the file won't be activated for a locale
|
|
until that locale has translated all of those required strings. The other rule is a percentage
|
|
completion rule: a certain percentage (configurable) of the strings IDs in the "en" file must
|
|
be translated in the file for a locale before it will be marked as active. We'll get into how
|
|
exactly this works later.
|
|
|
|
Translating with .ftl files
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The `Fluent file syntax`_ is well documented on the Fluent Project's site. We use "double hash" or
|
|
"group" comments to indicate strings required for activation. A group comment only ends when
|
|
another group comment starts however, so you should either group your required strings at the
|
|
bottom of a file, or also have a "not required" group comment. Here's an example:
|
|
|
|
.. code-block:: fluent
|
|
|
|
### File for example.html
|
|
|
|
## Required
|
|
example-page-title = The Page Title
|
|
example-page-desc = This page is a test.
|
|
|
|
##
|
|
example-footer = This string isn't as important
|
|
|
|
|
|
Any group comment (a comment that starts with "##") that starts with "Required" (case does not
|
|
matter) will start a required strings block, and any other group comment will end it.
|
|
|
|
Once you have your strings in your .ftl file you can place them in your template. We'll use the
|
|
above .ftl file for a simple Jinja template example:
|
|
|
|
.. code-block:: jinja
|
|
|
|
<!doctype html>
|
|
<html>
|
|
<head>
|
|
<title>{{ ftl('example-page-title') }}</title>
|
|
</head>
|
|
<body>
|
|
<h1>{{ ftl('example-page-title') }}</h1>
|
|
<p>{{ ftl('example-page-desc') }}</p>
|
|
<footer>
|
|
<p>{{ ftl('example-footer') }}</p>
|
|
</footer>
|
|
</body>
|
|
</html>
|
|
|
|
.. _Fluent file syntax: https://projectfluent.org/fluent/guide/
|
|
|
|
FTL String IDs
|
|
~~~~~~~~~~~~~~
|
|
|
|
Our convention for string ID creation is the following:
|
|
|
|
1. String IDs should be all lower-case alphanumeric characters.
|
|
2. Words should be separated with hyphens.
|
|
3. IDs should be prefixed with the name of the template file (e.g. ``firefox-new-skyline`` for ``firefox-new-skyline.html``)
|
|
4. If you need to create a new string for the same place on a page and to transition to it as it is translated, you can
|
|
add a version suffix to the string ID: e.g. ``firefox-new-skyline-main-page-title-v2``.
|
|
5. The ID should be as descriptive as possible to make sense to the developer, but could be anything as long as it adheres
|
|
to the rules above.
|
|
|
|
Using brand names
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
Common brand names are stored in a global `brands.ftl`_ file, in the form of `terms`_. Terms are useful for
|
|
keeping brand names separated from the rest of the translations, so that they can be managed in a consistent
|
|
way across all translated files, and also updated easily in a global context. In general the brand names in
|
|
this file remain in English and should not be translated, however locales still have the choice and control
|
|
to make adjustments should it suit their particular language.
|
|
|
|
.. code-block:: text
|
|
|
|
-brand-name = Firefox
|
|
|
|
about = About { -brand-name }.
|
|
update-successful = { -brand-name } has been updated.
|
|
|
|
.. important::
|
|
|
|
When adding a new term to ``brands.ftl``, the new term should also be manually added to the
|
|
`mozilla-l10n/www-l10n`_ repo for *all locales*. The reason for this is that if a term does not exist
|
|
for a particular locale, then it does not fall back to English like a regular string does. Instead,
|
|
the term variable name is displayed on the page.
|
|
|
|
.. _brands.ftl: https://github.com/mozilla/bedrock/blob/master/l10n/en/brands.ftl
|
|
.. _terms: https://projectfluent.org/fluent/guide/terms.html
|
|
.. _mozilla-l10n/www-l10n: https://github.com/mozilla-l10n/www-l10n
|
|
|
|
The ``ftl`` helper function
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
The ``ftl()`` function takes a string ID and returns the string in the current language,
|
|
or simplified english if the string isn't translated. If you'd like to use a different
|
|
string ID in the case that the primary one isn't translated you can specify that like this:
|
|
|
|
.. code-block:: python
|
|
|
|
ftl('primary-string-id', fallback='fallback-string-id')
|
|
|
|
When a fallback is specified it will be used only if the primary isn't translated in the current
|
|
locale. English locales (e.g. en-US, en-GB) will never use the fallback and will print the simplified
|
|
english version of the primary string if not overridden in the more specific locale.
|
|
|
|
You can also pass in replacement variables into the ``ftl()`` function for use with `fluent variables`_.
|
|
If you had a variable in your fluent file like this:
|
|
|
|
.. code-block:: fluent
|
|
|
|
welcome = Welcome, { $user }!
|
|
|
|
You could use that in a template like this:
|
|
|
|
.. code-block:: jinja
|
|
|
|
<h2>{{ ftl('welcome', user='Dude') }}<h2>
|
|
|
|
For our purposes these are mostly useful for things that can change, but which shouldn't involve
|
|
retranslation of a string (e.g. URLs or email addresses).
|
|
|
|
This helper is available in Jinja templates and Python code in views. For use in a view you should
|
|
always call it in the view itself:
|
|
|
|
.. code-block:: python
|
|
|
|
# views.py
|
|
from lib.l10n_utils import render
|
|
from lib.l10n_utils.fluent import ftl
|
|
|
|
def about_view(request):
|
|
ftl_files = 'mozorg/about'
|
|
hello_string = ftl('about-hello', ftl_files=ftl_files)
|
|
render(request, 'about.html', {'hello': hello_string}, ftl_files=ftl_files)
|
|
|
|
If you need to use this string in a view, but define it outside of the view itself, you can use the
|
|
``ftl_lazy`` variant which will delay evaluation until render time. This is mostly useful for defining
|
|
messages shared among several views in constants in a ``views.py`` or ``models.py`` file.
|
|
|
|
Whether you use this function in a Python view or a Jinja template it will always use the default
|
|
list of Fluent files defined in the ``FLUENT_DEFAULT_FILES`` setting. If you don't specify any additional
|
|
Fluent files via the ``fluent_files`` keyword argument, then only those default files will be used.
|
|
|
|
The ``ftl_has_messages`` helper function
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Another useful template tool is the ``ftl_has_messages()`` function. You pass it any number
|
|
of string IDs and it will return ``True`` only if all of those message IDs exist in the current
|
|
translation. This is useful when you want to add a new block of HTML to a page that is already
|
|
translated, but don't want it to appear untranslated on any page.
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% if ftl_has_messages('new-title', 'new-description') %}
|
|
<h3>{{ ftl('new-title') }}</h3>
|
|
<p>{{ ftl('new-description') }}</p>
|
|
{% else %}
|
|
<h3>{{ ftl('title') }}</h3>
|
|
<p>{{ ftl('description') }}</p>
|
|
{% endif %}
|
|
|
|
If you'd like to have it return true when any of the given message IDs exist in the translation
|
|
instead of requiring all of them, you can pass the optional ``require_all=False`` parameter and
|
|
it will do just that.
|
|
|
|
There is a version of this function for use in views called ``has_messages``. It works exactly the
|
|
same way but is meant to be used in the view Python code.
|
|
|
|
.. code-block:: python
|
|
|
|
# views.py
|
|
from lib.l10n_utils import render
|
|
from lib.l10n_utils.fluent import ftl, has_messages
|
|
|
|
def about_view(request):
|
|
ftl_files = 'mozorg/about'
|
|
if has_messages('about-hello-v2', 'about-title-v2',
|
|
ftl_files=ftl_files):
|
|
hello_string = ftl('about-hello-v2', ftl_files=ftl_files)
|
|
title_string = ftl('about-title-v2', ftl_files=ftl_files)
|
|
else:
|
|
hello_string = ftl('about-hello', ftl_files=ftl_files)
|
|
title_string = ftl('about-title', ftl_files=ftl_files)
|
|
|
|
render(request, 'about.html', {'hello': hello_string, 'title': title_string}, ftl_files=ftl_files)
|
|
|
|
.. _fluent variables: https://projectfluent.org/fluent/guide/variables.html
|
|
|
|
.. _specifying_fluent_files:
|
|
|
|
Specifying Fluent files
|
|
-----------------------
|
|
|
|
You have to tell the system which Fluent files to use for a particular template or view.
|
|
This is done in either the ``page()`` helper in a ``urls.py`` file, or in the call
|
|
to ``l10n_utils.render()`` in a view.
|
|
|
|
Using the ``page()`` function
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
If you just need to render a template, which is quite common for bedrock, you will probably
|
|
just add a line like the following to your ``urls.py`` file:
|
|
|
|
.. code-block:: python
|
|
|
|
urlpatterns = [
|
|
page('about', 'about.html'),
|
|
page('about/contact', 'about/contact.html'),
|
|
]
|
|
|
|
To tell this page to use the Fluent framework for l10n you just need to tell it which file(s)
|
|
to use:
|
|
|
|
.. code-block:: python
|
|
|
|
urlpatterns = [
|
|
page('about', 'about.html', ftl_files='mozorg/about'),
|
|
page('about/contact', 'about/contact.html', ftl_files=['mozorg/about/contact', 'mozorg/about']),
|
|
]
|
|
|
|
The system uses the first (or only) file in the list to determine which locales are active for that
|
|
URL. You can pass a string or list of strings to the ``ftl_files`` argument. The files you specify
|
|
can include the ``.ftl`` extension or not, and they will be combined with the list of default files
|
|
which contain strings for global elements like navigation and footer. There will also be files for
|
|
reusable widgets like the newsletter form, but those should always come last in the list.
|
|
|
|
Using the class-based view
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Bedrock includes a generic class-based view (CBV) that sets up l10n for you. If you need to do anything fancier
|
|
than just render the page, then you can use this:
|
|
|
|
.. code-block:: python
|
|
|
|
from lib.l10n_utils import L10nTemplateView
|
|
|
|
class AboutView(L10nTemplateView):
|
|
template_name = 'about.html'
|
|
ftl_files = 'mozorg/about'
|
|
|
|
Using that CBV will do the right things for l10n, and then you can override other useful methods
|
|
(e.g. ``get_context_data``) to do what you need. Also, if you do need to do anything fancy with
|
|
the context, and you find that you need to dynamically set the fluent files list, you can easily do
|
|
so by setting ``ftl_files`` in the context instead of the class attribute.
|
|
|
|
.. code-block:: python
|
|
|
|
from lib.l10n_utils import L10nTemplateView
|
|
|
|
class AboutView(L10nTemplateView):
|
|
template_name = 'about.html'
|
|
|
|
def get_context_data(self, **kwargs):
|
|
ctx = super().get_context_data(**kwargs)
|
|
ftl_files = ['mozorg/about']
|
|
if request.GET.get('fancy'):
|
|
ftl_files.append('fancy')
|
|
|
|
ctx['ftl_files'] = ftl_files
|
|
return ctx
|
|
|
|
A common case is needing to use FTL files when one template is used, but not with another. In this case
|
|
you would have some logic to decide which template to use in the ``get_template_names()`` method. You can
|
|
set the ``ftl_files_map`` class variable to a dict containing a map of template names to the list of
|
|
FTL files for that template (or a single file name if that's all you need).
|
|
|
|
.. code-block:: python
|
|
|
|
# views.py
|
|
from lib.l10n_utils import L10nTemplateView
|
|
|
|
# class-based view example
|
|
class AboutView(L10nTemplateView):
|
|
ftl_files_map = {
|
|
'about_es.html': ['about_es']
|
|
'about_new.html': ['about']
|
|
}
|
|
|
|
def get_template_names(self):
|
|
if self.request.locale.startswith('en'):
|
|
template_name = 'about_new.html'
|
|
elif self.request.locale.startswith('es'):
|
|
template_name = 'about_es.html'
|
|
else:
|
|
# FTL system not used
|
|
template_name = 'about.html'
|
|
|
|
return [template_name]
|
|
|
|
If you need for your URL to use multiple Fluent files to determine the full list of active locales,
|
|
for example when you are redesigning a page and have multiple templates in use for a single URL depending
|
|
on locale, you can use the `activation_files` parameter. This should be a list of FTL filenames
|
|
(or template file names if they are still using .lang) that should all be used when determining the full
|
|
list of translations for the URL. Bedrock will gather the full list for each file and combine them into a
|
|
single list so that the footer language switcher works properly.
|
|
|
|
Using in a view function
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Lastly there's the good old function views. These should use ``l10n_utils.render`` directly to render
|
|
the template with the context. You can use the ``ftl_files`` argument with this function as well.
|
|
|
|
.. code-block:: python
|
|
|
|
from lib.l10n_utils import render
|
|
|
|
def about_view(request):
|
|
render(request, 'about.html', {'name': 'Duder'}, ftl_files='mozorg/about')
|
|
|
|
Fluent File Configuration
|
|
-------------------------
|
|
|
|
In order for a Fluent file to be extracted through automation and sent out for localization,
|
|
it must first be configured to go through one or more distinct pipelines. This is controlled
|
|
via a set of configuration files:
|
|
|
|
- `Vendor`_, locales translated by an agency, and paid for by Marketing (locales covered by staff are also included in this group).
|
|
- `Pontoon`_, locales translated by Mozilla contributors.
|
|
- `Special templates`_, for locales with dedicated templates that don't go through the localization process (not currently used).
|
|
|
|
Each configuration file consists of a pre-defined set of locales for which each group is
|
|
responsible for translating. The locales defined in each file should not be changed without
|
|
first consulting the with L10n team, and such changes should not be a regular occurence.
|
|
|
|
To establish a localization strategy for a Fluent file, it needs to be included as a path
|
|
in one or more configuration files. For example:
|
|
|
|
.. code-block:: text
|
|
|
|
[[paths]]
|
|
reference = "en/mozorg/mission.ftl"
|
|
l10n = "{locale}/mozorg/mission.ftl"
|
|
|
|
You can read more about configuration files in the `L10n Project Configuration`_ docs.
|
|
|
|
.. important::
|
|
|
|
Path definitions in Fluent configuration files are not source order dependent. A broad
|
|
definition using a wild card can invalidate all previous path definitions for example.
|
|
Paths should be defined carefully to avoid exposing .ftl files to unintended locales.
|
|
|
|
Using a combination of vendor and pontoon configuration offers a flexible but specific set of
|
|
options to choose from when it comes to defining an l10n strategy for a page. The available
|
|
choices are:
|
|
|
|
#. Staff locales.
|
|
#. Staff + select vendor locales.
|
|
#. Staff + all vendor locales.
|
|
#. Staff + vendor + pontoon.
|
|
#. All pontoon locales (for non-marketing content only).
|
|
|
|
When choosing an option, it's important to consider that vendor locales have a cost associated
|
|
with them, and pontoon leans on the goodwill of our volunteer community. Typically, only
|
|
non-marketing content should go through Pontoon for all locales. Everything that is marketing
|
|
related should feature one of the staff/vendor/pontoon configurations.
|
|
|
|
.. _Vendor: https://github.com/mozilla/bedrock/blob/master/l10n/configs/vendor.toml
|
|
.. _Pontoon: https://github.com/mozilla/bedrock/blob/master/l10n/configs/pontoon.toml
|
|
.. _Special templates: https://github.com/mozilla/bedrock/blob/master/l10n/configs/special-templates.toml
|
|
.. _L10n Project Configuration: https://moz-l10n-config.readthedocs.io/
|
|
|
|
Fluent File Activation
|
|
----------------------
|
|
|
|
Fluent files are activated automatically when processed from the l10n team's repo
|
|
into our own based on a couple of rules.
|
|
|
|
1. If a fluent file has a group of required strings, all of those strings must be present in
|
|
the translation in order for it to be activated.
|
|
2. A translation must contain a minimum percent of the string IDs from the English file to be activated.
|
|
|
|
If both of these conditions are met the locale is activated for that particular Fluent file. Any view
|
|
using that file as its primary (first in the list) file will be available in that locale.
|
|
|
|
Deactivation
|
|
~~~~~~~~~~~~
|
|
|
|
If the automated system activates a locale but we for some reason need to ensure that this page remains
|
|
unavailable in that locale, we can add this locale to a list of deactivated locales in the metadata file
|
|
for that FTL file. For example, say we needed to make sure that the `mozorg/mission.ftl` file remained
|
|
inactive for German, even though the translation is already done. We would add ``de`` to the ``inactive_locales``
|
|
list in the ``metadata/mozorg/mission.json`` file:
|
|
|
|
.. code-block:: json
|
|
|
|
{
|
|
"active_locales": [
|
|
"de",
|
|
"fr",
|
|
"en-GB",
|
|
"en-US",
|
|
],
|
|
"inactive_locales": [
|
|
"de"
|
|
],
|
|
"percent_required": 85
|
|
}
|
|
|
|
This would ensure that even though ``de`` appears in both lists, it will remain deactivated on the site. We
|
|
could just remove it from the active list, but automation would keep attempting to add it back, so for now
|
|
this is the best solution we have, and is an indication of the full list of locales that have satisfied the rules.
|
|
|
|
Alternate Rules
|
|
~~~~~~~~~~~~~~~
|
|
|
|
It's also possible to change the percentage of string completion required for activation on a per-file basis. In
|
|
the same metadata file as above, if a ``percent_required`` key exists in the JSON data (see above) it will be used
|
|
as the minimum percent of string completion required for that file in order to activate new locales.
|
|
|
|
.. note::
|
|
|
|
Once a locale is activated for a Fluent file it will **NOT** be automatically deactivated, even if the
|
|
rules change. If you need to deactivate a locale you should follow the `Deactivation`_ instructions.
|
|
|
|
|
|
Activation Status
|
|
~~~~~~~~~~~~~~~~~
|
|
|
|
You can determine and use the activation status of a Fluent file in a view to make some decisions; what
|
|
template to render for example. The way you would do that is with the ``ftl_file_is_active`` function.
|
|
For example:
|
|
|
|
.. code-block:: python
|
|
|
|
# views.py
|
|
from lib.l10n_utils import L10nTemplateView
|
|
from lib.l10n_utils.fluent import ftl_file_is_active
|
|
|
|
# class-based view example
|
|
class AboutView(L10nTemplateView):
|
|
ftl_files_map = {
|
|
'about.html': ['about']
|
|
'about_new.html': ['about_new', 'about']
|
|
}
|
|
def get_template_names(self):
|
|
if ftl_file_is_active('mozorg/about_new'):
|
|
template_name = 'about_new.html'
|
|
else:
|
|
template_name = 'about.html'
|
|
|
|
return [template_name]
|
|
|
|
# function view example
|
|
def about_view(request):
|
|
if ftl_file_is_active('mozorg/about_new'):
|
|
template = 'mozorg/about_new.html'
|
|
ftl_files = ['mozorg/about_new', 'mozorg/about']
|
|
else:
|
|
template = 'about.html'
|
|
ftl_files = ['mozorg/about']
|
|
|
|
render(request, template, ftl_files=ftl_files)
|
|
|
|
Active Locales
|
|
~~~~~~~~~~~~~~
|
|
|
|
To see which locales are active for a particular .ftl file you can either look in
|
|
the metadata file for that .ftl file, which is the one with the same path but in
|
|
the ``metadata`` folder instead of a locale folder in the www-l10n repository. Or
|
|
if you'd like something a bit nicer looking and more convenient there is the
|
|
``active_locales`` management command:
|
|
|
|
.. code-block:: bash
|
|
|
|
$ ./manage.py l10n_update
|
|
|
|
.. code-block:: bash
|
|
|
|
$ ./manage.py active_locales mozorg/mission
|
|
|
|
|
|
.. code-block:: bash
|
|
|
|
There are 91 active locales for mozorg/mission.ftl:
|
|
- af
|
|
- an
|
|
- ar
|
|
- ast
|
|
- az
|
|
- be
|
|
- bg
|
|
- bn
|
|
...
|
|
|
|
You get an alphabetically sorted list of all of the active locales for that .ftl file.
|
|
You should run ``./manage.py l10n_update`` as shown above for the most accurate and
|
|
up-to-date results.
|
|
|
|
String extraction
|
|
-----------------
|
|
|
|
The string extraction process for both new .ftl content and updates to existing .ftl
|
|
content is handled through automation. On each commit to master a command is run that
|
|
looks for changes to the ``l10n/`` directory. If a change is detected, it will copy
|
|
those files into a new branch in `mozilla-l10n/www-l10n`_ and then a bot will open a
|
|
pull request containing those changes. Once the pull request has been reviewed and
|
|
merged by the L10n team, everything is done.
|
|
|
|
.. _mozilla-l10n/www-l10n: https://github.com/mozilla-l10n/www-l10n
|
|
|
|
Legacy
|
|
======
|
|
|
|
This section describes the legacy l10n system based on .lang files, which will
|
|
be frozen and no longer supported for new translations in January of 2020.
|
|
|
|
.lang files
|
|
-----------
|
|
|
|
Bedrock supports a workflow similar to gettext. You extract all the
|
|
strings from the codebase, then merge them into each locale to get
|
|
them translated.
|
|
|
|
The files containing the strings are called ".lang files" and end with
|
|
a ``.lang`` extension.
|
|
|
|
To extract all the strings from the codebase, run:
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_extract
|
|
|
|
If you'd only like to extract strings from certain files, you may optionally
|
|
list them on the command line:
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_extract bedrock/mozorg/templates/mozorg/contribute.html
|
|
|
|
Command line glob matching will work as well if you want all of the HTML files
|
|
in a directory, for example:
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_extract bedrock/mozorg/templates/mozorg/*.html
|
|
|
|
That will use gettext to get all the needed localizations from Python
|
|
and HTML files, and will convert the result into a series of .lang
|
|
files inside ``locale/templates``. This directory represents the
|
|
"reference" set of strings to be translated, and you are free to
|
|
modify or split up .lang files here as needed (just make sure they are
|
|
being referenced correctly, from the code, see
|
|
:ref:`Which .lang file should it use? <which-lang>`).
|
|
|
|
Once you have extracted .lang files locally, they can then be added via
|
|
pull request to the `mozilla.org l10n repository`_ for translation.
|
|
You can read the `full documentation`_ for more information on the
|
|
extraction workflow.
|
|
|
|
.. _using-lang:
|
|
|
|
Translating with .lang files
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
To translate a string from a .lang file, simply use the gettext interface.
|
|
|
|
In a jinja2 template:
|
|
|
|
.. code-block:: jinja
|
|
|
|
<div>{{ _('Hello, how are you?') }}<div>
|
|
|
|
<div>{{ _('<a href="%s">Click here</a>')|format('http://mozilla.org/') }}</div>
|
|
|
|
<div>{{ _('<a href="%(url)s">Click here</a>')|format(url='http://mozilla.org/') }}</div>
|
|
|
|
Note the usage of variable substitution in the latter examples. It is
|
|
important not to hardcode URLs or other parameters in the string.
|
|
jinja's `format` filter lets us apply variables outsite of the string.
|
|
|
|
You can provide a one-line comment to the translators like this:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{# L10n: "like" as in "similar to", not "is fond of" #}
|
|
{{ _('Like this:') }}
|
|
|
|
The comment will be included in the .lang files above the string to be
|
|
translated.
|
|
|
|
In a Python file, use ``lib.l10n_utils.dotlang._`` or
|
|
``lib.l10n_utils.dotlang._lazy``, like this:
|
|
|
|
.. code-block:: python
|
|
|
|
from lib.l10n_utils.dotlang import _lazy as _
|
|
|
|
sometext = _('Foo about bar.')
|
|
|
|
You can provide a one-line comment to the translators like this:
|
|
|
|
.. code-block:: python
|
|
|
|
# L10n: "like" as in "similar to", not "is fond of"
|
|
sometext = _('Like this:')
|
|
|
|
The comment will be included in the .lang files above the string to be
|
|
translated.
|
|
|
|
|
|
There's another way to translate content within jinja2 templates. If
|
|
you need a big chunk of content translated, you can put it all inside
|
|
a `trans` block.
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% trans %}
|
|
<div>Hello, how are you</div>
|
|
{% endtrans %}
|
|
|
|
{% trans url='http://mozilla.org' %}
|
|
<div><a href="{{ url }}">Click here</a></div>
|
|
{% endtrans %}
|
|
|
|
Note that it also allows variable substitution by passing variables
|
|
into the block and using template variables to apply them.
|
|
|
|
|
|
.. _which-lang:
|
|
|
|
Which .lang file should it use?
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Translated strings are split across several .lang files to make it
|
|
easier to manage separate projects and pages. So how does the system
|
|
know which one to use when translating a particular string?
|
|
|
|
* All translations from Python files are put into main.lang. This
|
|
should be a very limited set of strings and most likely should be
|
|
available to all pages.
|
|
* Templates always load `main.lang` and `download_button.lang`.
|
|
* Additionally, each template has its own .lang file, so a template at
|
|
`mozorg/firefox.html` would use the .lang file at
|
|
`<locale>/mozorg/firefox.lang`.
|
|
* Templates can override which .lang files are loaded. The above
|
|
global ones are always loaded, but instead of loading
|
|
`<locale>/mozorg/firefox.lang`, the template can specify a list of
|
|
additional lang files to load with a template block:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% add_lang_files "foo" "bar" %}
|
|
|
|
That will make the page load `foo.lang` and `bar.lang` in addition to
|
|
`main.lang` and `download_button.lang`.
|
|
|
|
When strings are extracted from a template, they are added to the
|
|
template-specific .lang file. If the template explicitly specifies
|
|
.lang files like above, it will add the strings to the first .lang
|
|
file specified, so extracted strings from the above template would go
|
|
into `foo.lang`.
|
|
|
|
You can similarly specify extra .lang files in your Python source as well.
|
|
Simply add a module-level constant in the file named `LANG_FILES`. The
|
|
value should be either a string, or a list of strings, similar to the
|
|
`add_lang_files` tag above.
|
|
|
|
.. code-block:: python
|
|
|
|
# forms.py
|
|
|
|
from lib.l10n_utils.dotlang import _
|
|
|
|
LANG_FILES = ['foo', 'bar']
|
|
|
|
sometext = _('Foo about bar.')
|
|
|
|
This file's strings would be extracted to `foo.lang`, and the lang files
|
|
`foo.lang`, `bar.lang`, `main.lang` and `download_button.lang` would be
|
|
searched for matches in that order.
|
|
|
|
l10n blocks
|
|
-----------
|
|
|
|
Bedrock also has a block-based translation system that works like the
|
|
``{% block %}`` template tag, and marks large sections of translatable
|
|
content. This should not be used very often; lang files are the
|
|
preferred way to translate content. However, there may be times when
|
|
you want to control a large section of a page and customize it
|
|
without caring very much about future updates to the English page.
|
|
|
|
A Localizers' guide to l10n blocks
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Let's look at how we would translate an example file from **English** to
|
|
**German**.
|
|
|
|
The English source template, created by a developer, lives under
|
|
`apps/appname/templates/appname/example.html` and looks like this:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% extends "base-protocol.html" %}
|
|
|
|
{% block content %}
|
|
<img src="someimage.jpg">
|
|
|
|
{% l10n foo, 20110801 %}
|
|
<h1>Hello world!</h1>
|
|
{% endl10n %}
|
|
|
|
<hr>
|
|
|
|
{% l10n bar, 20110801 %}
|
|
<p>This is an example!</p>
|
|
{% endl10n %}
|
|
{% endblock %}
|
|
|
|
The ``l10n`` blocks mark content that should be localized.
|
|
Realistically, the content in these blocks would be much larger. For a
|
|
short string like above, please use lang files. We'll use this trivial
|
|
code for our example though.
|
|
|
|
The ``l10n`` blocks are named and tagged with a date (in ISO format).
|
|
The date indicates the time that this content was updated and needs to
|
|
be translated. If you are changing trivial things, you shouldn't
|
|
update it. The point of l10n blocks is that localizers completely
|
|
customize the content, so they don't care about small updates.
|
|
However, you may add something important that needs to be added in the
|
|
localized blocks; hence, you should update the date in that case.
|
|
|
|
When the command ``./manage.py l10n_extract`` is run, it generates
|
|
the corresponding files in the ``locale`` folder (see below for more
|
|
info on this command).
|
|
|
|
The German version of this template is created at
|
|
``locale/de/templates/appname/example.html``. The contents of it are:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% extends "appname/example.html" %}
|
|
|
|
{% l10n foo %}
|
|
<h1>Hello world!</h1>
|
|
{% endl10n %}
|
|
|
|
{% l10n bar %}
|
|
<p>This is an example!</p>
|
|
{% endl10n %}
|
|
|
|
This file is an actual template for the site. It extends the main
|
|
template and contains a list of l10n blocks which override the content
|
|
on the page.
|
|
|
|
The localizer just needs to translate the content in the l10n blocks.
|
|
|
|
When the reference template is updated with new content and the date
|
|
is updated on an l10n block, the generated l10n file will simply add
|
|
the new content. It will look like this:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% extends "appname/example.html" %}
|
|
|
|
{% l10n foo %}
|
|
<h1>This is an English string that needs translating.</h1>
|
|
{% was %}
|
|
<h1>Dies ist ein English string wurde nicht.</h1>
|
|
{% endl10n %}
|
|
|
|
{% l10n bar %}
|
|
<p>This is an example!</p>
|
|
{% endl10n %}
|
|
|
|
Note the ``was`` block in ``foo``. The old translated content is in
|
|
there, and the new content is above it. The ``was`` content is always
|
|
shown on the site, so the old translation still shows up. The
|
|
localizer needs to update the translated content and remove the ``was``
|
|
block.
|
|
|
|
Generating the locale files
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_check
|
|
|
|
This command will check which blocks need to be translated and update
|
|
the locale templates with needed translations. It will copy the
|
|
English blocks into the locale files if a translation is needed.
|
|
|
|
You can specify a list of locales to update:
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_check fr
|
|
|
|
.. code-block:: console
|
|
|
|
$ ./manage.py l10n_check fr de es
|
|
|
|
Currency
|
|
--------
|
|
|
|
When dealing with currency, make a separate gettext wrapper, placing the amount
|
|
inside a variable. You should also include a comment describing the intent. For
|
|
example:
|
|
|
|
.. code-block:: jinja
|
|
|
|
{# L10n: Inserts a sum in US dollars, e.g. '$100'. Adapt the string in your translation for your locale conventions if needed, ex: %(sum)s US$ #}
|
|
{{ _('$%(sum)s')|format(sum='15') }}
|
|
|
|
CSS
|
|
---
|
|
|
|
If a localized page needs some locale-specific style tweaks, you can add the
|
|
style rules to the page's stylesheet like this:
|
|
|
|
.. code-block:: css
|
|
|
|
html[lang="it"] #features li {
|
|
font-size: 20px;
|
|
}
|
|
|
|
html[dir="rtl"] #features {
|
|
float: right;
|
|
}
|
|
|
|
If a locale needs site-wide style tweaks, font settings in particular, you can
|
|
add the rules to ``/media/css/l10n/{{LANG}}/intl.css``. Pages on Bedrock
|
|
automatically includes the CSS in the base templates with the `l10n_css` helper
|
|
function. The CSS may also be loaded directly from other Mozilla sites with such
|
|
a URL: ``//mozorg.cdn.mozilla.net/media/css/l10n/{{LANG}}/intl.css``.
|
|
|
|
*Open Sans*, the default font on mozilla.org, doesn't offer non-Latin glyphs.
|
|
``intl.css`` can have ``@font-face`` rules to define locale-specific fonts using
|
|
custom font families as below:
|
|
|
|
* *X-LocaleSpecific-Light*: Used in combination with *Open Sans Light*. The font
|
|
can come in 2 weights: normal and optionally bold
|
|
* *X-LocaleSpecific*: Used in combination with *Open Sans Regular*. The font can
|
|
come in 2 weights: normal and optionally bold
|
|
* *X-LocaleSpecific-Extrabold*: Used in combination with *Open Sans Extrabold*.
|
|
The font weight is 800 only
|
|
|
|
Here's an example of ``intl.css``:
|
|
|
|
.. code-block:: css
|
|
|
|
@font-face {
|
|
font-family: X-LocaleSpecific-Light;
|
|
font-weight: normal;
|
|
font-display: swap;
|
|
src: local(mplus-2p-light), local(Meiryo);
|
|
}
|
|
|
|
@font-face {
|
|
font-family: X-LocaleSpecific-Light;
|
|
font-weight: bold;
|
|
font-display: swap;
|
|
src: local(mplus-2p-medium), local(Meiryo-Bold);
|
|
}
|
|
|
|
@font-face {
|
|
font-family: X-LocaleSpecific;
|
|
font-weight: normal;
|
|
font-display: swap;
|
|
src: local(mplus-2p-regular), local(Meiryo);
|
|
}
|
|
|
|
@font-face {
|
|
font-family: X-LocaleSpecific;
|
|
font-weight: bold;
|
|
font-display: swap;
|
|
src: local(mplus-2p-bold), local(Meiryo-Bold);
|
|
}
|
|
|
|
@font-face {
|
|
font-family: X-LocaleSpecific-Extrabold;
|
|
font-weight: 800;
|
|
font-display: swap;
|
|
src: local(mplus-2p-black), local(Meiryo-Bold);
|
|
}
|
|
|
|
Localizers can specify locale-specific fonts in one of the following ways:
|
|
|
|
* Choose best-looking fonts widely used on major platforms, and specify those
|
|
with the ``src: local(name)`` syntax
|
|
* Find a best-looking free Web font, add the font files to ``/media/fonts/``,
|
|
and specify those with the ``src: url(path)`` syntax
|
|
* Create a custom Web font to complement missing glyphs in *Open Sans*, add the
|
|
font files to ``/media/fonts/l10n/``, and specify those with the
|
|
``src: url(path)`` syntax. `M+ 2c <http://mplus-fonts.osdn.jp/about-en.html>`_
|
|
offers various international glyphs and looks similar to Open Sans, while
|
|
`Noto Sans <https://www.google.com/get/noto/>`_ is good for the bold and
|
|
italic variants. You can create subsets of these alternative fonts in the WOFF
|
|
and WOFF2 formats using a tool found on the Web. See `Bug 1360812
|
|
<https://bugzilla.mozilla.org/show_bug.cgi?id=1360812>`_ for the Fulah (ff)
|
|
locale's example
|
|
|
|
Developers should use the ``.open-sans`` mixin instead of ``font-family: 'Open
|
|
Sans'`` to specify the default font family in CSS. This mixin has both *Open
|
|
Sans* and *X-LocaleSpecific* so locale-specific fonts, if defined, will be
|
|
applied to localized pages. The variant mixins, ``.open-sans-light`` and
|
|
``.open-sans-extrabold``, are also available.
|
|
|
|
Staging Copy Changes
|
|
--------------------
|
|
|
|
The need will often arise to push a copy change to production before the new
|
|
copy has been translated for all locales. To prevent locales not yet translated
|
|
from displaying English text, you can use the ``l10n_has_tag`` template
|
|
function. For example, if the string "Firefox benefits" needs to be changed to
|
|
"Firefox features":
|
|
|
|
.. code-block:: jinja
|
|
|
|
{% if l10n_has_tag('firefox_products_headline_spring_2016') %}
|
|
<h1>{{ _('Firefox features') }}</h1>
|
|
{% else %}
|
|
<h1>{{ _('Firefox benefits') }}</h1>
|
|
{% endif %}
|
|
|
|
This function will check the .lang file(s) of the current page for the tag
|
|
``firefox_products_headline_spring_2016``. If it exists, the translation for
|
|
"Firefox features" will be displayed. If not, the pre-existing translation for
|
|
"Firefox benefits" will be displayed.
|
|
|
|
When using ``l10n_has_tag``, be sure to coordinate with the localization team to
|
|
decide on a good tag name. Always use underscores instead of hyphens if you need
|
|
to visually separate words.
|
|
|
|
All
|
|
===
|
|
|
|
Locale-specific Templates
|
|
-------------------------
|
|
|
|
While the ``l10n_has_tag`` or ``ftl_has_messages`` template functions are great in small doses,
|
|
they don't scale particularly well. A template filled with conditional copy can be
|
|
difficult to comprehend, particularly when the conditional copy has associated
|
|
CSS and/or JavaScript.
|
|
|
|
In instances where a large amount of a template's copy needs to be changed, or
|
|
when a template has messaging targeting one particular locale, creating a
|
|
locale-specific template may be a good choice.
|
|
|
|
Locale-specific templates function simply by naming convention. For example, to
|
|
create a version of ``/firefox/new.html`` specifically for the ``de`` locale,
|
|
you would create a new template named ``/firefox/new.de.html``. This template
|
|
can either extend ``/firefox/new.html`` and override only certain blocks, or be
|
|
entirely unique.
|
|
|
|
When a request is made for a particular page, bedrock's rendering function
|
|
automatically checks for a locale-specific template, and, if one exists, will
|
|
render it instead of the originally specified (locale-agnostic) template.
|
|
|
|
.. NOTE::
|
|
|
|
Creating a locale-specific template for en-US was not possible when this
|
|
feature was introduced, but it is now. So you can create your en-US-only
|
|
template and the rest of the locales will continue to use the default.
|
|
|
|
.. IMPORTANT::
|
|
|
|
Note that the presence of an L10n template (e.g.
|
|
``locale/de/templates/firefox/new.html``) will take precedence over
|
|
a locale-specific template in bedrock.
|
|
|
|
|
|
Specifying Active Locales in Views
|
|
----------------------------------
|
|
|
|
Normally we rely on activation tags in our translation files (.lang files)
|
|
to determine in which languages a page will be available. This will almost always
|
|
be what we want for a page. But sometimes we need to explicitly state the locales
|
|
available for a page. The `impressum` page for example is only available in German
|
|
and the template itself has German hard-coded into it since we don't need it to be
|
|
translated into any other languages. In cases like these we can send a list of locale
|
|
codes with the template context and it will be the final list. This can be accomplished
|
|
in a few ways depending on how the view is coded.
|
|
|
|
For a plain view function, you can simply pass a list of locale codes to `l10n_utils.render`
|
|
in the context using the name `active_locales`. This will be the full list of available
|
|
translations. Use `add_active_locales` if you want to add languages to the existing list:
|
|
|
|
.. code-block:: python
|
|
|
|
def french_and_german_only(request):
|
|
return l10n_utils.render(request, 'home.html', {'active_locales': ['de', 'fr'])
|
|
|
|
If you don't need a custom view and are just using the `page()` helper function in your `urls.py`
|
|
file, then you can similarly pass in a list:
|
|
|
|
.. code-block:: python
|
|
|
|
page('about', 'about.html', active_locales=['en-US', 'es-ES']),
|
|
|
|
Or if your view is even more fancy and you're using a Class-Based-View that inherits from `LangFilesMixin`
|
|
(which it must if you want it to be translated) then you can specify the list as part of the view Class
|
|
definition:
|
|
|
|
.. code-block:: python
|
|
|
|
class MyView(LangFilesMixin, View):
|
|
active_locales = ['zh-CN', 'hi-IN']
|
|
|
|
Or in the `urls.py` when using a CBV:
|
|
|
|
.. code-block:: python
|
|
|
|
url(r'about/$', MyView.as_view(active_locales=['de', 'fr'])),
|
|
|
|
The main thing to keep in mind is that if you specify `active_locales` that will be the full list of
|
|
localizations available for that page. If you'd like to add to the existing list of locales generated
|
|
from the lang files then you can use the `add_active_locales` name in all of the same ways as
|
|
`active_locales` above. It's a list of locale codes that will be added to the list already available.
|
|
This is useful in situations where we would have needed the l10n team to create an empty .lang file with
|
|
an active tag in it because we have a locale-specific-template with text in the language hard-coded into
|
|
the template and therefore do not otherwise need a .lang file.
|
|
|
|
Development
|
|
-----------
|
|
|
|
In local development environments and on demo servers all ``l10n_has_tag`` calls
|
|
evaluate to true. If the content has not been translated it will display
|
|
the English strings.
|
|
|
|
To test l10n locally you can set ``DEV=False`` in your ``.env`` file.
|
|
|
|
If you are running your local server you will need to restart it after altering
|
|
your ``.env`` file.
|
|
|
|
.. _mozilla.org l10n repository: https://github.com/mozilla-l10n/www.mozilla.org/
|
|
.. _full documentation: https://mozilla-l10n.github.io/documentation/products/mozilla_org/
|