DjangoBot: #celery

result.ready()

from celery.result import AsyncResult

result = AsyncResult(task_id)

ah, so I have to save my task_id on my own table

there is no, "Registry, give me all task ids"

no, there's not. if you need that, you have to implement it yourself

it's optimized around the common case

got it

thanks

please share if you implement it in a generic way :)

or not so generic, whatever. could be nice as an example

sure, i will

Celery is taking up a few hundred MB. Any idea why?

Or how I can find out? Bringing my server to a crawl

kill'd it. wasn't processing any jobs.

would love to find out why that happened

abadr: running with DEBUG on?

asksol: Django? one server yes, one server no

with a few hundred MB, are you talking RAM?

yes

just today it jumped up... didn't even have any jobs

it's been running the background for a week or so

if settings.DEBUG is on, that would probably happen at some point

why?

because it logs queries

hm

ok

there could be other reasons, but I haven't heard about it before

I see that I'm also on an old version. Will upgrade to 1.0 and see if that helps.

right, that would do it then

the old version checks the periodic task table almost every second

== a connection.queries the size of the sun

i see

the new version doesn't use the database for periodic tasks at all

just listens to amqp?

no, there's a new service called celerybeat

I saw those scripts installed to /usr/local/bin

it's a centralized, service so it doesn't have to share state, so there's no chance for race conditions

and also, it doesn't have to poll the table every second :)

am I supposed to install celeryinit to init.d?

no, it's just for creating tables when not using python

read the changelog: http://ask.github.com/celery/changelog.html

k

run celeryd like before, just if you use periodic tasks, you need to run celerybeat

if you don't need periodic tasks, don't run it

and don't run more than one celerybeat servers

can you foresee celery having any problems handling two instances of the same app running on the same machine?

-s

one is in debug, one isn't

not sure

but it's pretty theoretical at this stage, with an older version and no similar reports

yep

You should watch it a bit more closely for a while

asksol: sorry, but I'm a little confused. What's the correct way to daemonize celery now if I don't use periodic tasks? Changelog seems to say celerybeat, you say otherwise :)

yeah I will keep an eye on it

that doesn't have anything to do with periodic tasks. The --detach argument has been removed because it was buggy

and there was no fix in sight

so use start-stop-daemon on debian/bsd

or supervisord on anything else

there's example init scripts here:http://github.com/ask/celery/tree/master/contrib/debian/init.d/

and example supervisord configuration here: http://github.com/ask/celery/tree/master/contri...

supervisord is pretty great, as it also make sure it's always running

oh, I see. thanks

asksol: start-stop-daemon isn't working. it thinks it's working, and a new pid gets written to the pidfile, but then nothing is left running and nothing is written to the log