DjangoBot: #celery

there's whoever creates the task (probably a view in your django app)

there's the AMQP broker (probably rabbitmq, unless you're masochistic)

and then there's the celery workers.

or more accurately, celery daemons, each of which can run several celery workers at once.

there's nothing at all that says that any of these three elements need to share a machine

you can have your appserver fire off tasks which get queued on rabbitmq on another machine

and yet another machine has a celery daemon with workers who are processing that queue's tasks.

so yes, the run() only gets executed wherever the worker is at.

idangazit: ok, that's pretty much how I understood it. The magic celery works is just the passing of args from the requesting function to the workers, and potentially dealing with the result, right?

the magic celery works is in hiding the plumbing of AMQP from you :)

but yeah, you enqueue tasks with some args, magically, somewhere else there is a worker which runs some logic on that data

idangazit: what does adding celery to my INSTALLED_APPS buy me?

um, the ability to enqueue tasks using the same db django is using?

there's probably more but my brain is kinda offline.

tbh I've never used celery without django, so I don't really know what is different about using celery w/o django.

auto detection of tasks

donspaulding: you don't need the task on the client

you can execute it by name

or just register a dummy task with that name

adding to INSTALLED_APPS just adds the celery models to the app registry

so the tables are created on syncdb, etc

I should create an easy way to execute a task using just the name

asksol: sorry, I'm sure this is in the docs, but what does celery store in the DB? Just results?

yeah

just results, if using the database backend that is

and you can use it for messaging to if you use ghettoq

too

asksol: so, calling celery.registry.tasks.register() is only really useful if I'm making use of Django in the worker, right

?

no

it's what maps a task name to a task class

asksol: does a task need to be registered before it is .delay()ed?

so the message sent contains {"task_name": "celery.task.ping", "args": [], "kwargs": {}}

yeah

but you could use celery.messaging.TaskPublisher directly

or you could make a dummy task in the client

this is a work around, I think we should have a way to do this easily

asksol: I think I'm just coming up with too many stupid questions because I've not implemented enough yet. I'll come back when I've tried more things.

asksol: I'm not actually running into problems yet, just trying to understand everything I've read about celery up till now.

hehe, right

brb