Task Manager

Spirit ships with several tasks that run asynchronously. The supported task managers to run these tasks are Huey, and Celery.

While Spirit can run without a task manager, some of the tasks may not run, as they are too expensive to run as part of the request-response cycle (web server), or they are require to be run periodically.

Functionality that requires a task manager includes: search indexation.


Read the Huey docs for Django.



pip install django-spirit[huey]


Set Huey as the Spirit task manager:

# settings.py (dev.py or prod.py)

# extend installed apps with +=


The settings/prod.py and settings/dev.py files include sample configuration for Huey. The prod.py settings set Redis as the backend, and the dev.py settings set SQLite as the backend. Redis is recommended for production environments, and it’s a good option to use as the Django cache as well.

How to install Redis depends on the OS, for example it can be installed as a Snap package in Ubuntu, or it can be run with docker. So, it’s out of the scope of these docs.

Run the tasks consumer

To start the task manager, run:

python manage.py run_huey

This can be run with supervisord, or systemd in production.

The caveats of SQLite as a backend

SQLite only works in single server instances, where both the web-server and the task manager (Huey) are running in the same machine, sharing the same SQLite data base.

For sufficiently large forum instances, it’s recommended to run the task manager on its own machine, while the web-server can scale up and down independently. This means Redis must be used instead of SQLite.

Scaling Huey

I’m not aware of how well Huey scales horizontally. It seems it may just work fine. However, all Huey instances but one must disable the periodic task functionality to avoid running periodic tasks in more than one machine. This can be done by passing the --no-periodic parameter to the task manager command.


Read the Celery docs for Django.



pip install django-spirit[celery]


Set Celery as the Spirit task manager:

# mysite/settings/settings.py (dev.py or prod.py)

ST_TASK_MANAGER = 'celery'

Import Celery at Django startup:

# mysite/__init__.py

from .celery import app as celery_app
__all__ = ('celery_app',)

Run the tasks consumer

To start the task manager, run:

celery -A mysite worker -l info

This can be run with supervisord, or systemd in production.

To start the periodic task manager, run:

celery -A mysite beat -l info

Note this will just enqueue tasks that will later be consumed by the worker.

Celery does not work

Celery likes to break all the things in every major version. The celery configuration (including prod.py, dev.py, and mysite/celery.py) were tested on v4.4.7.

I’m not a Celery user. These docs and the Celery support are a community effort. Please, send a PR if something breaks.