I just use the defaults actually

i'm not sure what you can set there even, NODENAME, and high_memory_watermark at least

:) yup I always go with default guest:guest

Is celerymon good to install ?

almost 12PM here good night

nice. tests are passing when using django trunk

celery-users is now the second hit for "celery" on google

asksol: have you really tested celery much with transactions in django mysql innodbs?

then I suppose, since the API looks similar, is a Task just a Thread? :)

then even further, if you had to aggregate 1 million feeds, would you just dump a task for every feed?

my current approach was to run a periodic check for feeds which needed sync'd, and then run a bunch of threads to work on those

so I suppose I could do the same, periodic task + sending a bunch of tasks

And I suppose, my last question, do workers need strings/whatever passed as args, or can objects work ?

is there a way to have a single server work on multiple queues?

mulka: http://ask.github.com/celery/faq.html#can-i-sen...

davidcramer: we do exactly that

or, there's some other optimizations involved

like it applies n tasks at a time

I can send you some code

it should be open sourced soon

bbl

umm what us the best/recommended way to have celeryd start when the server boots? (Ubuntu)

d0ugal: not sure if its the best way, but I would start it using start-stop-daemon in /etc/rc.local

how does a celery worker decide which queue to pull from if there are multiple? Does each "process" just round robin or something?

d0ugal: http://github.com/ask/celery/tree/master/contri...

mulka: rabbitmq decides

bbl

asksol: but if a worker is assigned to multiple queues...

its not like each worker has its own queue to pull from, right?

asksol: just a rough draft based on some of your recipes :) http://www.pastethat.com/AkxJX

asksol, FYI there seems to be a typo here; http://celeryproject.org/tutorials/otherqueues.... I think toproot should be taproot?

hey

yashh: hi

I see this in my celery logs

AMQP Listener: Connection Error: [Errno 111]

may celery is having trouble connecting to your AMQP server? have you double checked the settings?

is celery actually working, or not?

mulka: sorry, yes celery is working

but once in a while I see that in my log

yashh: I don't really have a better answer for you. someone else might know more

hey np. I dont think its major but just something.

d0ugal: yeah, thanks! already fixed in the latest version btw

development docs that is

released celery 0.8.4

just a small maintenance release

it now pukes out a warning if --detach is used, and some typos fixed

is there an easy way to get at the number of tasks left in a queue via the celery or carrot apis ? It wasn't clear to me in the docs..

madness: well, not programatically

but it will be easier with celerymon

rabbitmqctl list_queues -p myvhost

asksol: yup, I figure I could just parse that out. I have a task that spits more tasks into the queue for a crawler once every now and then, but don't want it to continue doing so when the queue is bigger than 50k

When I use the Supervised flag w/mange.py celeryd, it doesn't process any tasks and I think the workers keep dying

I guess I'll just run it without -S for now...

I think this is asked 12 times a day, but is there an easy way to see current queued tasks?

asksol: amqp as a tombstone backend, that warning, i wanted to confirm what "receive the result once" means :)

i set the concurrency to 4, and it shows 5 procs, kinda of odd ;)

yay, celery running on demo site :D

ooo, error emails :D

and i suppose i should fork/commit this back, but for now ill just post here: http://celeryproject.org/reference/celery.task....

kwargs/args should really be optional I think

in fact i will probably fork tomorrow

i think i have some useful additions to commit back

davidcramer: there's no easy way to see queued tasks until celerymon is done

5 processes, one main process and 4 pool workers?

you can see what they do in ps listings if you install the "setproctitle" module

thanks!

amqp backend means you send the result as a message, so you can only consume that result once

so if you do result.get() in two different processes, you will have a deadlock

or deadlock is probably not the right term

plan for 1.2 is to have an option to cache the result in a result backend

so send back result with amqp, e.g. store it in redis when consumed

I'm loving Celery :D

we've gotten it all moved into tasks and its working amazing

so any heavy task no longer happens on the web

should make scaling a lot easier as well

I'm assuming I could just have a project+celery install on any server, then one rabbitmq install for it to communicate with? Easy scaling, ya?

:)

indeed

cool

I did have one transactional issue

and I have had the same issue without celery, so I would say its more of a flaw in Django

for some reason django doesnt commit the save() transaction here

(at the bottom)

hi

without rabbitq, how configure the celery and django

pls, its urgent

i thought rabbitmq was a requirement

or is that entirely based on the backend setting?

celery with databse is it possible

gnperumal: http://ask.github.com/celery/tutorials/otherque...

Hi

I have a question regarding signals in celery

is the signal 'task_postrun' send every time a request runs ?

request?

it's sent when after a task has been executed

executed by the worker, or using apply()/ALWAYS_EAGER locally (happens in celery.execute.trace)

but this is in 1.0, in 0.x.x, it was only sent when executed by the worker