bedrock/docs/task-queue.rst

.. This Source Code Form is subject to the terms of the Mozilla Public
.. License, v. 2.0. If a copy of the MPL was not distributed with this
.. file, You can obtain one at https://mozilla.org/MPL/2.0/.

.. _task-queue:

==========
Task queue
==========

As part of the 2024 Wagtail CMS work, we have added a task queue to process
longer-running jobs in the background.

The queue is supplied by ``django-rq``, which sits on top of ``rq`` and uses
Redis as the storage backend.

Once infra has been set up fully to support this, documentation will be expanded,
but for local development you can start a worker to process jobs in the queue with:

.. code-block:: bash

        make run-local-task-queue

Note that you will need Redis running on its default port of 6379. Redis is
started by default when using Dockerized Bedrock locally.
Add background task queue support and use it for CMS image generation (#14617) * Add django-rq to the project, with initial configuration for two redis-backed queues if redis is available * Add Django-rq to project, including link from (now necessary) Django admin's userlinks * Update Docker support to run a local worker node for RQ * Add Sentry integration for Redis and for RQ * Add a make command to run the worker queue Temporarily loads env vars from .env, looking for REDIS_URL * Make a new setting that makes it unambiguous whether a task queue is available * Add helper that will enqueue a task or immediately execute it, depending on whether the task queue is available Note that if it finds a queue is available it will put a job in there, regardless of whether the queue is being processed by a worker * Switch image-rendition pre-generation to use the task queue, if available * Add initial, basic, docs about the task queue * Improve logging around task queue usage * Fix formatting of Makefile Thanks @janbrasna! * Typo fixes in documentation Thanks @janbrasna! 2024-06-04 13:27:34 +03:00			`.. This Source Code Form is subject to the terms of the Mozilla Public`
			`.. License, v. 2.0. If a copy of the MPL was not distributed with this`
			`.. file, You can obtain one at https://mozilla.org/MPL/2.0/.`

			`.. _task-queue:`

			`==========`
			`Task queue`
			`==========`

			`As part of the 2024 Wagtail CMS work, we have added a task queue to process`
			`longer-running jobs in the background.`

			The queue is supplied by ``django-rq``, which sits on top of ``rq`` and uses
			`Redis as the storage backend.`

			`Once infra has been set up fully to support this, documentation will be expanded,`
			`but for local development you can start a worker to process jobs in the queue with:`

			`.. code-block:: bash`

			`make run-local-task-queue`

			`Note that you will need Redis running on its default port of 6379. Redis is`
			`started by default when using Dockerized Bedrock locally.`