Celery vs Cron vs a Django Management Command: Pick the Simplest Tool

Overview

A common pattern in Django projects: someone needs a nightly cleanup or a weekly report, and Celery, a broker, a beat process and a result backend get added to the stack. Six months later the team is debugging a stuck beat scheduler for a job that could have been one line in a crontab.

Celery is a good tool, but it adds moving parts you have to deploy, monitor and upgrade. This post gives concrete criteria for when you do not need it.

1. Start With What the Job Actually Needs

Ask these questions about the work:

• Is it triggered by a user action (a request) or by the clock?
• Does the caller need to return quickly while the work continues?
• Does it need retries, rate limiting or fan-out across many workers?
• How many runs per day, and how long does each take?
• What happens if one run is skipped or runs twice?

If the trigger is purely the clock, the work is a single process, and a missed run can simply be caught up by the next one, you probably do not need a queue.

2. Cron Plus a Management Command

Put the logic in a Django management command and let the operating system schedule it. The command is testable, runnable by hand, and has no dependency on a broker.

# app/management/commands/purge_old_exports.py
from datetime import timedelta

from django.core.management.base import BaseCommand
from django.utils import timezone

from app.models import ExportFile


class Command(BaseCommand):
    help = "Delete export files older than the retention period."

    def add_arguments(self, parser):
        parser.add_argument("--days", type=int, default=30)

    def handle(self, *args, **options):
        cutoff = timezone.now() - timedelta(days=options["days"])
        deleted, _ = ExportFile.objects.filter(created_at__lt=cutoff).delete()
        self.stdout.write(f"Deleted {deleted} rows")

# crontab
15 2 * * * cd /srv/app && /srv/app/.venv/bin/python manage.py purge_old_exports --days 30 >> /var/log/purge.log 2>&1

This is a good fit for nightly cleanup, report generation, syncing a file once an hour, or sending a daily digest to a modest number of users.

Things to handle yourself: overlapping runs (use flock or a database advisory lock if a run can exceed the interval), alerting when the job fails (cron only emails if mail is configured), and running on exactly one host when you have several servers. On containers, a Kubernetes CronJob or your platform's scheduled-job feature does the same thing without a long-lived process.

3. When a Queue Is the Right Answer

Choose Celery (or another task queue) when the work is triggered by a request and must not block it, or when you need queue features:

• A user uploads a file and you must return a response immediately while processing continues.
• You need automatic retries with backoff for a flaky third-party API.
• You need to fan out thousands of small units of work across many workers.
• You need rate limiting per downstream service, priorities or separate queues.
• Work must continue after the web process is redeployed or restarted.

Here is the same kind of work as a task, triggered from a view:

from celery import shared_task
from django.db import transaction

from app.models import Document


@shared_task(autoretry_for=(ConnectionError,), retry_backoff=True, max_retries=3)
def extract_text(document_id: int) -> None:
    document = Document.objects.get(pk=document_id)
    document.text = run_extraction(document.file.path)
    document.save(update_fields=["text"])


def upload_view(request):
    document = Document.objects.create(file=request.FILES["file"])
    transaction.on_commit(lambda: extract_text.delay(document.pk))
    ...

Note transaction.on_commit: it prevents the worker from picking up the task before the row is visible.

4. A Middle Path: Management Command Triggered by a Queue

You do not have to choose one extreme. If you already run Celery for request-driven work, a periodic job can still be a management command that you call from cron. Keeping the logic in one function that both a command and a task can call lets you change your mind later without a rewrite.

Conversely, if you only need Celery for one scheduled job, that is a strong sign to use cron instead. Celery beat is one more process that must run as a singleton, and running two copies by accident causes duplicate runs.

5. A Quick Decision Table

• Runs on a schedule, single process, tolerates a missed run: cron plus a management command.
• Runs on a schedule but must run on exactly one of many hosts: a platform scheduler (such as a Kubernetes CronJob) or a lock.
• Triggered by a user action, short enough to do in the request: just do it in the request.
• Triggered by a user action, slow or unreliable: a task queue.
• Needs retries, fan-out, priorities or rate limits: a task queue.

Also consider the middle options your platform may already offer, such as a managed scheduler or a one-off job runner. They give you scheduling and isolation without a broker.

Conclusion

Use the least machinery that meets the requirements. A management command is easy to test and run by hand, cron or a platform scheduler is enough for clock-driven work, and Celery earns its place when requests must return quickly or work needs retries and scale.

If you are unsure whether your Django project needs a task queue, or want to simplify a stack that has grown too many moving parts, I can help review it. Contact me.

Additional Resources

• Writing custom django-admin commands
• Celery periodic tasks
• Django transaction.on_commit