DjangoBot: #celery

i am wondering i there would be a reason the output of rabbitmqctl list_queues -p / name messages consumers would be incorrect?

for instance i get: celeryctl_worker37 0 1

celery 458 37

celeryctl_worker19 0 1

celery 458 37*

but i don't believe there are 458 jobs in the queue

I have an interesting problem with a periodic task and the beat server. I'm running in Django, python manage.py celeryd -B and the periodic task I have defined executes fine, no problems at all. I stopped celeryd and started again, this time without the beat option e.g. python manage.py celeryd and the funny thing is that the periodic task is still going! It keeps executing

This wasn't what I was expecting, I was really expecting that it wouldn't execute because there was no beat server. Any suggestions? I only have two celery worker servers, both on seperate machines and they both show . beat -> OFF in the terminal window

tim__ - maybe a stray beat process still running?

Thanks - I was just about to say I think I've found it...

When I run ps aux | grep celeryd I can see some celeryd -B processes still there. I must of killed the wrong process before

tim__: celery 1.0.1 will be released soon, fixes a bug with the -B option

that is that the celerybeat process is not shutdown properly

asksol: jesper was tweeting this morning about 2.4 support in case you didn't notice

yeah

working with him now

I was waiting for him to test it

ah. cool

fixed a bug in master with eta tasks not being executed

that is, it's not present in 1.0, just recent changes

asksol: Just wanted to let you know I figured out how to do a periodic task for multiple instances of a class

code here works without modification needed for celery 1.0.0 : http://pastebin.com/KiypHWtV

Can anyone explain the difference between python manage.py celeryd -B VS python manage.py celerybeat? I expected they would be the same, but they behave differently. With a four core machine, running celeryd - B I end up with five processes (presumably four normal celeryd and one beat server). When I run celerybeat I end up with only one celerybeat server, which I'd expect, BUT the celerybeat server doesn't seem to execute the

periodic tasks, where as the celerd -B option does.

OK, don't worry - just as I was writing this I figured out what I was doing wrong :-)

For the record, just realised they ARE doing the same thing, but when I run a celerybeat only I have no worker running to process the tasks, whereas with celeryd -B I get both workers and the beat server which is why it looks like its working /DOH

asksol: what's the biggest deployment of celery you know of in terms of number of daemons? wondering if there's any known issues in growing the amount

How do the number of daemons / workers relate to the queues, especially in Django where the queues / bindings are specified in settings.py so they are shared? Does each worker listen to all queues? Or if there were four workers and two queues, do two listen to one queue and two to the other?