зеркало из https://github.com/mozilla/treeherder.git
Bug 1466084 - Docs: Switch to MkDocs and Material theme (#4565)
Switches from Sphinx to MkDocs, since it: * supports Markdown natively without requiring hacks like `eval_rst` * validates inline links, ensuring that they are not broken * has a more pleasant live-reloading dev-server * supports the nicer looking mkdocs-material theme * is a third of the size of Sphinx (including deps) The theme change is now possible since Read the Docs have just started supporting use of custom themes (previously they would override the theme and use `readthedocs` theme regardless).
This commit is contained in:
Ed Morley
GitHub
Родитель
bcb0bc2d7c
Коммит
65287fed30
|
|
@ -0,0 +1,7 @@
|
|||
# https://docs.readthedocs.io/en/latest/config-file/index.html
|
||||
version: 2
|
||||
python:
|
||||
install:
|
||||
- requirements: requirements/docs.txt
|
||||
mkdocs:
|
||||
configuration: mkdocs.yml
|
||||
|
|
@ -32,7 +32,7 @@ matrix:
|
|||
- flake8 --show-source
|
||||
- isort --check-only --diff --quiet
|
||||
- git grep -El '^#!/.+\b(bash|sh)\b' | xargs shellcheck
|
||||
- make html
|
||||
- mkdocs build
|
||||
|
||||
- env: python2-tests-main
|
||||
language: python
|
||||
|
|
|
|||
29
Makefile
29
Makefile
|
|
@ -1,29 +0,0 @@
|
|||
# Makefile for Sphinx documentation
|
||||
|
||||
SPHINXOPTS = -n -W
|
||||
SPHINXBUILD = sphinx-build
|
||||
BUILDDIR = .build-docs
|
||||
SOURCEDIR = docs
|
||||
|
||||
# User-friendly check for sphinx-build
|
||||
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
|
||||
$(error Sphinx not found. Run 'pip install -r requirements/docs.txt' first.)
|
||||
endif
|
||||
|
||||
.PHONY: help clean html livehtml
|
||||
|
||||
help:
|
||||
@echo "Please use \`make <target>' where <target> is one of"
|
||||
@echo " html to make standalone HTML files"
|
||||
@echo " livehtml to start the live-reloading web server"
|
||||
|
||||
clean:
|
||||
rm -rf ./$(BUILDDIR)/
|
||||
|
||||
html:
|
||||
$(SPHINXBUILD) -b html $(SPHINXOPTS) "$(SOURCEDIR)" "$(BUILDDIR)"
|
||||
@echo
|
||||
@echo "Build finished. The HTML pages are in $(BUILDDIR)."
|
||||
|
||||
livehtml:
|
||||
sphinx-autobuild -b html --poll $(SPHINXOPTS) $(SOURCEDIR) $(BUILDDIR) -B -p 8001
|
||||
|
|
@ -24,9 +24,9 @@ The steps to run only the UI are provided [here](https://treeherder.readthedocs.
|
|||
#### Links
|
||||
|
||||
Visit our project tracking Wiki at:
|
||||
https://wiki.mozilla.org/EngineeringProductivity/Projects/Treeherder
|
||||
<https://wiki.mozilla.org/EngineeringProductivity/Projects/Treeherder>
|
||||
|
||||
Visit our **readthedocs** page for other setup and configuration at:
|
||||
https://treeherder.readthedocs.io/
|
||||
<https://treeherder.readthedocs.io>
|
||||
|
||||
File any bugs you may encounter [here](https://bugzilla.mozilla.org/enter_bug.cgi?product=Tree+Management&component=Treeherder).
|
||||
|
|
|
|||
|
|
@ -9,10 +9,8 @@ Vagrant.require_version ">= 2.1.5"
|
|||
Vagrant.configure("2") do |config|
|
||||
# webpack-dev-server
|
||||
config.vm.network "forwarded_port", guest: 5000, host: 5000, host_ip: "127.0.0.1"
|
||||
# Django runserver/gunicorn
|
||||
# Django runserver/gunicorn/mkdocs serve
|
||||
config.vm.network "forwarded_port", guest: 8000, host: 8000, host_ip: "127.0.0.1"
|
||||
# Docs dev server
|
||||
config.vm.network "forwarded_port", guest: 8001, host: 8001, host_ip: "127.0.0.1"
|
||||
# MySQL
|
||||
config.vm.network "forwarded_port", guest: 3306, host: 3308, host_ip: "127.0.0.1"
|
||||
# Elasticsearch
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ Or for more control, run each tool individually:
|
|||
|
||||
NB: If running flake8 from outside of the VM, ensure you are using the same version as used on Travis (see `requirements/dev.txt`).
|
||||
|
||||
- [isort](https://github.com/timothycrosley/isort) (checks the [Python import style](code_style.html#python-imports)):
|
||||
- [isort](https://github.com/timothycrosley/isort) (checks the [Python import style](code_style.md#python-imports)):
|
||||
|
||||
To run interactively:
|
||||
|
||||
|
|
@ -80,11 +80,9 @@ In the below example we're mapping VM port 3306 (MySQL's default port) to host p
|
|||
config.vm.network "forwarded_port", guest: 3306, host: 3308, host_ip: "127.0.0.1"
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. note::
|
||||
|
||||
<!-- prettier-ignore -->
|
||||
!!! note
|
||||
Any forwarded ports will block usage of that port on the host OS even if there isn't a service running inside the VM talking to it.
|
||||
```
|
||||
|
||||
With MySQL exposed at port 3308 you can connect to it from your host OS with the following credentials:
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ In addition:
|
|||
```
|
||||
|
||||
The quickest way to correct import style locally is to let isort make the changes for you - see
|
||||
[running the tests](common_tasks.html#running-the-tests).
|
||||
[running the tests](common_tasks.md#running-the-tests).
|
||||
|
||||
Note: It's not possible to disable isort wrapping style checking, so for now we've chosen the
|
||||
most deterministic [wrapping mode](https://github.com/timothycrosley/isort#multi-line-output-modes)
|
||||
|
|
@ -33,7 +33,7 @@ to reduce the line length guess-work when adding imports, even though it's not t
|
|||
|
||||
We use Prettier for JS/JSX formatting and the [Airbnb](https://github.com/airbnb/javascript)
|
||||
guide for non-style related best practices. Both are validated using ESlint (see Validating
|
||||
Javascript in the [Installation section](installation.html#validating-javascript)).
|
||||
Javascript in the [Installation section](installation.md#validating-javascript)).
|
||||
|
||||
Prettier is also used to format JSON/CSS/HTML/Markdown/YAML. However these are not supported
|
||||
by ESLint, so instead are validated using Prettier's CLI. To manually check their formatting
|
||||
|
|
|
|||
|
|
@ -7,10 +7,10 @@
|
|||
|
||||
```bash
|
||||
> pip install -r requirements/docs.txt
|
||||
> make livehtml
|
||||
> mkdocs serve
|
||||
```
|
||||
|
||||
- Visit <http://127.0.0.1:8001> to view the docs.
|
||||
- Visit <http://127.0.0.1:8000> to view the docs.
|
||||
- Source changes will result in automatic rebuilds and browser page reload.
|
||||
|
||||
## Updating package.json
|
||||
|
|
@ -37,4 +37,4 @@ To add a new repository, the following steps are needed:
|
|||
- Restart any running gunicorn/celery processes.
|
||||
|
||||
For more information on adding a new GitHub repository, see
|
||||
[Adding a GitHub repository](submitting_data.html#adding-a-github-repository).
|
||||
[Adding a GitHub repository](submitting_data.md#adding-a-github-repository).
|
||||
|
|
|
|||
28
docs/conf.py
28
docs/conf.py
|
|
@ -1,28 +0,0 @@
|
|||
import sphinx_rtd_theme
|
||||
from recommonmark.parser import CommonMarkParser
|
||||
from recommonmark.transform import AutoStructify
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
project = u'Treeherder'
|
||||
copyright = u'Mozilla and other contributors'
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
exclude_patterns = ['.build-docs']
|
||||
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
|
||||
source_parsers = {
|
||||
'.md': CommonMarkParser,
|
||||
}
|
||||
|
||||
source_suffix = ['.rst', '.md']
|
||||
|
||||
|
||||
def setup(app):
|
||||
# Enable additional recommonmark features:
|
||||
# https://recommonmark.readthedocs.io/en/latest/auto_structify.html
|
||||
app.add_transform(AutoStructify)
|
||||
|
|
@ -0,0 +1 @@
|
|||
--8<-- "README.md"
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
Welcome to the Treeherder documentation!
|
||||
========================================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Getting Started
|
||||
|
||||
installation
|
||||
code_style
|
||||
common_tasks
|
||||
backend_tasks
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Working with the Server
|
||||
|
||||
pulseload
|
||||
seta
|
||||
rest_api
|
||||
submitting_data
|
||||
retrieving_data
|
||||
data_validation
|
||||
troubleshooting
|
||||
testcases
|
||||
admin
|
||||
|
|
@ -8,14 +8,14 @@
|
|||
|
||||
If you only want to hack on the frontend, see the UI Development section below. If you want to hack on the backend or work full-stack, see the [Server and Full-stack Development](#server-and-full-stack-development) section.
|
||||
|
||||
# UI Development
|
||||
## UI Development
|
||||
|
||||
To get started:
|
||||
|
||||
- Install [Node.js] and [Yarn] (see [package.json] for known compatible versions, listed under `engines`).
|
||||
- Run `yarn install` to install all dependencies.
|
||||
|
||||
## Running the standalone development server
|
||||
### Running the standalone development server
|
||||
|
||||
The default development server runs the unminified UI and fetches data from the
|
||||
production site. You do not need to set up the Vagrant VM unless making backend changes.
|
||||
|
|
@ -26,11 +26,11 @@ production site. You do not need to set up the Vagrant VM unless making backend
|
|||
$ yarn start
|
||||
```
|
||||
|
||||
```eval_rst
|
||||
.. note::
|
||||
|
||||
Any action you take, such as classifying a job, will affect the live production front-end of Treeherder so we recommend developing against `stage` (details below) unless there's something data-specific that must be addressed on production.
|
||||
```
|
||||
<!-- prettier-ignore -->
|
||||
!!! note
|
||||
Any action you take, such as classifying a job, will affect the live production
|
||||
front-end of Treeherder, so we recommend developing against `stage` (details below)
|
||||
unless there's something data-specific that must be addressed on production.
|
||||
|
||||
- The server will perform an initial build and then watch for new changes. Once the server is running, you can navigate to: <http://localhost:5000> to see the UI.
|
||||
|
||||
|
|
@ -48,7 +48,7 @@ production site. You do not need to set up the Vagrant VM unless making backend
|
|||
|
||||
This will run the unminified UI using `<url>` as the service domain.
|
||||
|
||||
## Validating JavaScript
|
||||
### Validating JavaScript
|
||||
|
||||
We run our JavaScript code in the frontend through [ESLint] to ensure
|
||||
that new code has a consistent style and doesn't suffer from common
|
||||
|
|
@ -68,9 +68,9 @@ Or to automatically fix issues found (where possible):
|
|||
$ yarn lint --fix
|
||||
```
|
||||
|
||||
See the [code style](code_style.html#ui) section for more details.
|
||||
See the [code style](code_style.md#ui) section for more details.
|
||||
|
||||
## Running the unit tests
|
||||
### Running the unit tests
|
||||
|
||||
Currently, most of the unit tests for the UI are run with [Karma] and [Jasmine]. However, we are
|
||||
migrating from [Karma] to [Jest]. So new tests should be written in [Jest]. React components are tested with [enzyme].
|
||||
|
|
@ -95,14 +95,14 @@ $ yarn jest:watch
|
|||
|
||||
The tests will perform an initial run and then re-execute each time a project file is changed.
|
||||
|
||||
# Server and Full-stack Development
|
||||
## Server and Full-stack Development
|
||||
|
||||
To get started:
|
||||
|
||||
- Install [Virtualbox] and [Vagrant] (latest versions recommended).
|
||||
- Linux only: An nfsd server is required. You can install this on Ubuntu by running `apt-get install nfs-common nfs-kernel-server`
|
||||
|
||||
## Setting up Vagrant
|
||||
### Setting up Vagrant
|
||||
|
||||
- Open a shell, cd into the root of the Treeherder repository, and type:
|
||||
|
||||
|
|
@ -131,10 +131,10 @@ To get started:
|
|||
- For the full list of available Vagrant commands (for example, suspending the VM when you are finished for the day),
|
||||
see their [command line documentation](https://www.vagrantup.com/docs/cli/).
|
||||
|
||||
- If you just wish to [run the tests](common_tasks.html#running-the-tests),
|
||||
- If you just wish to [run the tests](common_tasks.md#running-the-tests),
|
||||
you can stop now without performing the remaining steps.
|
||||
|
||||
## Starting a local Treeherder instance
|
||||
### Starting a local Treeherder instance
|
||||
|
||||
- Start a gunicorn instance inside the Vagrant VM, to serve the static UI and API requests:
|
||||
|
||||
|
|
@ -161,7 +161,7 @@ To get started:
|
|||
|
||||
- Visit <http://localhost:5000> in your browser (NB: port has changed). Note: There will be no data to display until the ingestion tasks are run.
|
||||
|
||||
## Building the minified UI with Vagrant
|
||||
### Building the minified UI with Vagrant
|
||||
|
||||
If you would like to view the minified production version of the UI with Vagrant, follow this step:
|
||||
|
||||
|
|
@ -174,21 +174,7 @@ If you would like to view the minified production version of the UI with Vagrant
|
|||
Once the build is complete, the minified version of the UI will now be accessible at
|
||||
<http://localhost:8000> (NB: port 8000, unlike above).
|
||||
|
||||
## Validating JavaScript
|
||||
|
||||
We run our JavaScript code in the frontend through [eslint] to ensure
|
||||
that new code has a consistent style and doesn't suffer from common
|
||||
errors. Eslint will run automatically when you build the JavaScript code
|
||||
or run the development server. A production build will fail if your code
|
||||
does not match the style requirements.
|
||||
|
||||
To run eslint by itself, you may run the lint task:
|
||||
|
||||
```bash
|
||||
$ yarn lint
|
||||
```
|
||||
|
||||
## Running the ingestion tasks
|
||||
### Running the ingestion tasks
|
||||
|
||||
Ingestion tasks populate the database with version control push logs, queued/running/completed jobs & output from log parsing, as well as maintain a cache of intermittent failure bugs. To run these:
|
||||
|
||||
|
|
@ -203,14 +189,14 @@ Ingestion tasks populate the database with version control push logs, queued/run
|
|||
|
||||
- Then in a new terminal window, run `vagrant ssh` again, and follow the steps from the [loading pulse data](pulseload.md) page.
|
||||
|
||||
## Ingesting a single push (at a time)
|
||||
### Ingesting a single push (at a time)
|
||||
|
||||
```eval_rst
|
||||
.. warning::
|
||||
With the end of life of buildbot, this command is no longer able to ingest jobs.
|
||||
For now after running it, you will need to manually follow the steps from the
|
||||
:doc:`loading pulse data<pulseload>` page.
|
||||
```
|
||||
<!-- prettier-ignore -->
|
||||
!!! warning
|
||||
With the end of life of buildbot, this command is no longer able to ingest jobs.
|
||||
|
||||
For now after running it, you will need to manually follow the steps from the
|
||||
[loading pulse data](pulseload.md) page.
|
||||
|
||||
Alternatively, instead of running a full ingestion task, you can process just
|
||||
the jobs associated with any single push generated in the last 4 hours
|
||||
|
|
@ -225,7 +211,7 @@ vagrant ~/treeherder$ ./manage.py ingest_push mozilla-inbound 63f8a47cfdf5
|
|||
If running this locally, replace `63f8a47cfdf5` with a recent revision (= pushed within
|
||||
the last four hours) on mozilla-inbound.
|
||||
|
||||
## Ingesting a range of pushes
|
||||
### Ingesting a range of pushes
|
||||
|
||||
It is also possible to ingest the last N pushes for a repository:
|
||||
|
||||
|
|
|
|||
|
|
@ -34,9 +34,9 @@ PULSE_URL="amqp://foo:bar@pulse.mozilla.org:5671/?ssl=1"
|
|||
|
||||
### 3. Read Pushes
|
||||
|
||||
```eval_rst
|
||||
.. note:: Be sure your Vagrant environment is up-to-date. Reload it and run ``vagrant provision`` if you're not sure.
|
||||
```
|
||||
<!-- prettier-ignore -->
|
||||
!!! note
|
||||
Be sure your Vagrant environment is up-to-date. Reload it and run ``vagrant provision`` if you're not sure.
|
||||
|
||||
`ssh` into Vagrant, then set your config environment variable:
|
||||
|
||||
|
|
@ -81,7 +81,7 @@ That's it! With those processes running, you will begin ingesting Treeherder
|
|||
data. To see the data, you will need to run the Treeherder UI and API.
|
||||
See [Running the unminified UI with Vagrant] for more info.
|
||||
|
||||
[running the unminified ui with vagrant]: ui/installation.html#running-the-unminified-ui-with-vagrant
|
||||
[running the unminified ui with vagrant]: installation.md#server-and-full-stack-development
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
|
|
@ -142,10 +142,14 @@ celery -A treeherder worker -B -Q pushlog,store_pulse_jobs,store_pulse_resultset
|
|||
- The `store_pulse_jobs` queue will ingest all the jobs from the exchanges
|
||||
specified in `job_sources` (or `PULSE_JOB_SOURCES`).
|
||||
|
||||
```eval_rst
|
||||
.. note:: Any job that comes from **Pulse** that does not have an associated push will be skipped.
|
||||
.. note:: It is slightly confusing to see ``store_pulse_resultsets`` there. It is there for legacy reasons and will change to ``store_pulse_pushes`` at some point.
|
||||
```
|
||||
<!-- prettier-ignore -->
|
||||
!!! note
|
||||
Any job that comes from Pulse that does not have an associated push will be skipped.
|
||||
|
||||
<!-- prettier-ignore -->
|
||||
!!! note
|
||||
It is slightly confusing to see ``store_pulse_resultsets`` there. It is there
|
||||
for legacy reasons and will change to ``store_pulse_pushes`` at some point.
|
||||
|
||||
## Posting Data
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Retrieving Data
|
||||
|
||||
The [Python client](rest_api.html#python-client) also has some convenience
|
||||
The [Python client](rest_api.md#python-client) also has some convenience
|
||||
methods to query the Treeherder API.
|
||||
|
||||
Here's a simple example which prints the start timestamp of all the
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ do one of the following:
|
|||
|
||||
1. For GitHub repos: [Adding a GitHub Repository](#adding-a-github-repository)
|
||||
|
||||
2. For Mercurial repos: [Add a new Mercurial repository](common_tasks.html#add-a-new-mercurial-repository)
|
||||
2. For Mercurial repos: [Add a new Mercurial repository](common_tasks.md#add-a-new-mercurial-repository)
|
||||
|
||||
## Using Pulse
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,49 @@
|
|||
docs_dir: 'docs'
|
||||
site_dir: '.build-docs'
|
||||
strict: true
|
||||
use_directory_urls: false
|
||||
|
||||
site_name: 'Treeherder Docs'
|
||||
site_url: 'https://treeherder.readthedocs.io'
|
||||
repo_name: 'mozilla/treeherder'
|
||||
repo_url: 'https://github.com/mozilla/treeherder'
|
||||
|
||||
# https://squidfunk.github.io/mkdocs-material/getting-started/#configuration
|
||||
theme:
|
||||
name: 'material'
|
||||
|
||||
# https://squidfunk.github.io/mkdocs-material/getting-started/#extensions
|
||||
markdown_extensions:
|
||||
- admonition
|
||||
- codehilite:
|
||||
guess_lang: false
|
||||
- toc:
|
||||
permalink: true
|
||||
# Fixes fenced code blocks within lists.
|
||||
- 'pymdownx.superfences':
|
||||
disable_indented_code_blocks: True
|
||||
# Allows index.md to include the repo root README.
|
||||
- 'pymdownx.snippets':
|
||||
check_paths: True
|
||||
# Required so that 2 space indents within lists work the same as
|
||||
# GitHub and Prettier's markdown implementation.
|
||||
- mdx_truly_sane_lists
|
||||
|
||||
# http://www.mkdocs.org/user-guide/writing-your-docs/#configure-pages-and-navigation
|
||||
nav:
|
||||
- About: 'index.md'
|
||||
- Getting Started:
|
||||
- Installation: 'installation.md'
|
||||
- Code Style: 'code_style.md'
|
||||
- Common tasks: 'common_tasks.md'
|
||||
- Backend tasks: 'backend_tasks.md'
|
||||
- Working with the Server:
|
||||
- Loading Pulse data: 'pulseload.md'
|
||||
- SETA: 'seta.md'
|
||||
- REST API: 'rest_api.md'
|
||||
- Submitting Data: 'submitting_data.md'
|
||||
- Retrieving Data: 'retrieving_data.md'
|
||||
- Schema Validation: 'data_validation.md'
|
||||
- Troubleshooting: 'troubleshooting.md'
|
||||
- Treeherder Test Cases: 'testcases.md'
|
||||
- Administrating Treeherder: 'admin.md'
|
||||
|
|
@ -2,8 +2,6 @@
|
|||
# since doc generation does not need all of the requirements in common.txt.
|
||||
# This file is used by Read the Docs, Travis and when building locally.
|
||||
|
||||
Sphinx==1.8.4
|
||||
sphinx-autobuild==0.7.1
|
||||
sphinx-rtd-theme==0.4.2
|
||||
recommonmark==0.5.0
|
||||
commonmark==0.8.1
|
||||
mkdocs==1.0.4
|
||||
mkdocs-material==3.3.0
|
||||
mdx_truly_sane_lists==1.2
|
||||
|
|
|
|||
|
|
@ -101,6 +101,9 @@ fi
|
|||
echo '-----> Running pip install'
|
||||
pip install --require-hashes -r requirements/common.txt -r requirements/dev.txt \
|
||||
| sed -e '/^Requirement already satisfied:/d'
|
||||
# Installing separately since we don't specify sub-deps and hashes for docs dependencies.
|
||||
pip install -r requirements/docs.txt \
|
||||
| sed -e '/^Requirement already satisfied:/d'
|
||||
|
||||
if [[ "$(geckodriver --version 2>&1)" != *"${GECKODRIVER_VERSION}"* ]]; then
|
||||
echo '-----> Installing geckodriver'
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче