DjangoBot: #celery

celery just sits for me.. doesn't take any of the tasks out of the queue

any suggestions?

what happens when you run celeryd

heedly^

otherjacob: it runs mmy periodic jobs, but the Tasks are never ran

can the box you're running celeryd talk to your broker outside of celery?

ya

hmpth

Hi, I'm just trying out celery for the first time with Django and Rabbit and I'm having a problem with the Django example from here http://ask.github.com/celery/getting-started/fi...

I can complete the first couple of sections - Configuring Django and Running the Celery worker server, but when I try the Defining and Executing Tasks section I hit some errors.

Basically, I've created a tasks.py in clifton.fetcher, when I use the python interactive command line and type from clifton.fetcher import tasks...

I get the following error ImportError: No module named celeryconfig

I'm guessing this is because I'm not properly referencing the settings.py? Any suggestions on where I'm going wrong or what I should import first would be really appreciated. Thanks!

make sure to run 'PYTHONPATH="." python'

when you start the interpreter

OK, thanks - I'll try that

otherwise it doesn't know where to find the file celeryconfig.py

Running python manage.py shell has got me past the import error - I think I'm getting the settings from my settings.py

Thanks for the help.

hello - just discovered celery this morning - installing... very excited :)

i'm porting a GAE app back to "normal python"/Django

my need is the Task Queue feature of GAE and its "deferred library"

is rabbitMQ _mandatory_ ?

jonnygo: no. There are a few others you can use

this is what first pages of doc seems to say, yes

my need is simple

i think storing tasks in DB is enough

CELERY_BACKEND = "database" means this ?

jonnygo: yeah. I guess you found the faq? http://celeryproject.org/docs/faq.html :)

jonnygo: yeah. see http://celeryproject.org/docs/configuration.html

thx for the RTFM link :) I must confess I didn't get it yet :) Will read this - sorry for the noise

first code snippet there

no problem

actually, I misunderstood. celery_backend is the setting for storing the info about tasks. this is what you want http://ask.github.com/celery/tutorials/otherque...

jonnygo: CELERY_BACKEND is for storing results

ah

padt to the rescue

asksol: this is what I saw - and lead me to this channel

in the meanwhile I discovered the other-queues page :)

just wondering why I need a carrot now :)

it's the messaging library

replacing rabbitmq ?

(sort of)

asksol: perhaps it's time to add a "what's the realtion between carrot and celery" question to the faq? Also, it would be nice to format the faq with the index of questions at the top

jonnygo: carrot talks to rabbitmq

or any amqp server I think

so I still need rabbitmq :)

no, carrot is an abstraction over amqplib

my initial question was to drop it :)

carrot supports messaging using amqplib/pika, stomp, redis and database

jonnygo: to be fair, there is a lot of concepts :)

so you're not the first

so it's a library to abstract away the AMQP parts, so we have freedom to use something else. like you using database

i'm building my first prototype following on "natural" documentation (so using rabbitmq)

i think it's overkill for my needs - but i _might_ understand it this way :)

(i need to delay an XML download+parse tasks)

hehe

the goal is that you shouldn't have to know that much about it until you need it though

so maybe I failed ;(

don't worry - i'm a bit dull :)

heedly: what platform?

hi, I'm looking at using celery, but I would like to verify if it fits my use case. I need something on the backend that will perform HTTP requests and reply those requests if it fails and / or report back that something (perhaps unrecoverable) occurred.

if not easily (and I mean very easily) I'll just make something from scratch really fast)

should fit well for that description

not sure what you mean by "reply those requests"

if set settings.CELERY_SEND_TASK_ERROR_EMAILS = True, celeryd will send e-mails to the ADMINS about failed tasks

if you set(*

a failed task is simply a task that raises an exception

I'm guessing that report back means the result of the request

which could be None if it failed. Or an exception

or whatever the request returned

would celery be able to reschedule the task if it failed? I suppose that could be the coder's onus to ensure it's rescheduled depending on the error correct

yes, there's Task.retry

k

and I need to setup rabbitMP in order to use celery? Or some other thing like that

rabbitmq, redis or any of the databases supported by Django

mysql would work then right?

however, rabbitmq is definitely recommended for serious use

yeah mysql works

ehh this isn't needed to be huge, just need it done fast... I can worry about scaling later

then you use rabbitmq later

you can just drop it in at any stage (have to migrate messages though), so no worry

k, then I'm having trouble trying to figure out how to configure celeryd to run without it... I get AMQP listener connection error's

did you read the otherqueues tutorial?

no

am doing now

are you using this with a django project?

yes

ok then

have it running it loooks like

so if I wanted to have a task triggers (say a http API to be hit) when objects are created / modified should I create the task in the signal handler or in save() / delete() ?

what's you guys opinion

signal handler always for save and delete

this is because if you delete more than one object, any overridden save method won't be called (afair)

or I think that was the issue, padt: do you remember what the issue was?

soemthing like that.. I'll poke around the django bts for my bug