Home › Python › Celery background jobs AGENTS.md

Celery background jobs AGENTS.md

AGENTS.md for Celery 5 task queues in Python/Django: idempotent tasks, retries, time limits, and safe enqueueing.

AGENTS.mdPython

Copy AGENTS.md

How to use

Save this as AGENTS.md in your repository root, commit it, and your agent will pick it up automatically. Edit the commands to match your project.

Note: Review and test before use. Provided as is, without warranty. See Terms.

# AGENTS.md: Celery tasks

## Commands
- Worker: `celery -A config worker -l info` · Beat: `celery -A config beat -l info` · Tests: `pytest -q`

## Writing tasks
- Tasks live in `<app>/tasks.py`, decorated `@shared_task(bind=True)`. Name them explicitly if they're referenced by string.
- Pass ids and primitives, never ORM objects or large payloads. Re-fetch the record inside the task.
- Tasks must be idempotent: running twice gives the same result (check state before acting, use unique constraints, or an idempotency key).
- Retries for transient errors only: `autoretry_for=(httpx.TransportError,)`, `retry_backoff=True`, `retry_jitter=True`, `max_retries=5`. Don't retry on validation errors.
- Set `soft_time_limit`/`time_limit` on anything that calls the network.
- For tasks that must not be lost on worker crash, use `acks_late=True` (and make them idempotent).
- Enqueue after the DB transaction commits: `transaction.on_commit(lambda: send_receipt.delay(order.id))` (or `delay_on_commit` where available), so workers don't read uncommitted rows.
- Long workflows: chains/groups/chords, each step small.
- Periodic tasks defined in code (beat schedule) or django-celery-beat, not hard-coded crons on servers.
- Log with the task id; never log secrets or full payloads with PII.

## Testing
Call the task function directly (`send_receipt.run(order_id)`) or use `task_always_eager` in tests only; mock external calls.

## Don't
Don't purge queues or revoke tasks in shared environments.