DjangoBot: #celery

the exact time requirement is not implemented yet

I've mentioned before it shouldn't be that hard to do

you just need a way to convert strings like "3:00" to a datetime object

I would be happy if you could implement that :)

there's probably a library around to do that

then you have to make a PeriodicTask that executes on a fixed time instead of on an interval

w00t! i think this works!

underseapilot: awesome!

i'm sensing a 1.0.1 release

underseapilot: It works with the HUP signal as well

underseapilot: please verify HUP, and close http://github.com/ask/celery/issues#issue/68 if it works

will do, though might not finish testing till tomorrow.

sure

will probably have to test these changes for a few days

exactly.

luckily no one is processing credit card transactions with celery, yet

:)

or, I hope :/

the plan is to be robust enough to be able to do that, so we definitely need some functional tests of the kind you're writing

agreed. though not sure how i'm gonna do that yet. :)

internally we fire up a celeryd and test against that.

maybe use the worker_ready signal to tell when the worker has started

and the task_received signal to know the task has been received

and I'll add any additional signals you need

or maybe using events

but it's okay to fire up a Worker in another process? ( know how, but seems different than the other tests )

alternative is to start worker -> send task -> hopefully sleep long enough before you shut down

you should do it via exec() then

not using multiprocessing

or, that is, not using fork()

because multiprocessing has some bugs lurking around that path

ok

asksol, this change http://github.com/ask/celery/commit/fa0a4d4763f... relies on current_process() from multiprocessing

and looks like the second call isn't fully qualified.

hi all - wondering if there was support for scheduled tasks (not PeriodicTasks, but one-off future tasks)

rlotun: there's an eta kwarg for tasks

padt: brilliant, thanks!

np

padt: I also sent an email to the google group but realized it might be faster to come here first, so ignore the double question ;-)

heh. good thinking

usually depends on the time of day if it's something Ask has to answer. We're in cet+1

you work for Opera too?

yeah. Not on the same stuff though. I'm just the pedant that goes through the docs every once in a while

that's cool. I work at a small startup in London called TweetDeck. celery is perfect for our needs and it's great that it exists.

that's the adobe air thingy? I think I tried that one

the adobe runtime was a bit iffy on linux at the time unfortunately

yeah, though we also have an iphone client and others in the works

you might want to give it another spin, I think those issues have been fixed ;-)

cool

at the moment I share an office with the guy that maintains the opera twitter widget though, so we'll see :)

come to think of it, I think I have a t shirt even. Was there a tweetdeck one from startupswag?

yeah actually

Then I've probably advertized for you guys in the office!

awesome! we can send more, and some tweeting birds too if you'd like ;-)

you should definitely give us a shout if you're ever in London

Cool. Will do

asksol: looking at the eta arg.. .i go down the path and see it end up in TaskPublisher, where its sent as part of message_data to the carrot Publisher. ...

not sure what is missing then...

could it be covertly implemented already?

asksol: gotta go feed my boys before they devour the furniture. let me know about eta (not the Basque separatists, you know the one i mean)

Hi there. I just got celery working great with Django and now I want to deploy it on some production sites. I want to know the best approach to this. I use virtualenv's for my sites on production, should I install seperate celery packages and run separate instances of it for each website, or am I better off to just run a single one and install the package under the root python install?

Ooh, I guess one issue I just encountered is that celery doesn't seem to be respecting it's default max_retries of 3 .. Mine has retried 7 times now. I can paste some code . if someone is alive :- )

Oh, I need to learn more about RabbitMQ (I bookarked asksol's article he linked!) .. it seems after restarting my rabbitmq-server, it's smoother now. Looks like I had a bunch of instances "caught up" from all my testing.

Hmm, it still keeps retrying infinietly.

infinitely*

bartek- maybe you want to start celeryd with --discard once?

man im in a totally different world of programming when you can restart a service and it fires off the backlog it had

awesome :)

Still trying to get celery to run tasks on Windows. celeryd's logging tells me the taskpool gets created, but the task never completes. Working off a test script that asksol gave me yesterday, here's my current traceback. Does anyone know what's going on here? http://pastie.org/829783

is that really the same file? b/c the traceback references line number that don't exist.

use x.apply_async(MicrParse(), ...) in that case

you're using the class as Target, so calling it means MicrParse.__new__(3)

so you have to instantiate it to make it use MicrParse.__call__(3)

urm. MicrParse().__call__(3)

but I think celery.loaders.detect_loader is the source of your problem

the real problem that is, the __new__ error is just because the test script is wrong

donspaulding: do you use "project.settings" as settings module?

underseapilot: not the same file, but the error is the same with the file I pasted. Nice catch ;-)

asksol: yes

asksol: changing it to MicrParse() gets me further. Now I've got an exception in my run(), will fix that before continuing the test.

donspaulding: don't use project.settings ;)

that's the bug

there's an issue about it at github

make celeryd cd into the project dir maybe

asksol: yeah, I read through that bug when you pasted it yesterday, I'm just not smart enough to see it.

here's my most recent exception http://pastie.org/829783

underseapilot: and yes, those are the actual files, unmodified ;-)

donspaulding, very good then. ;)

don't run such a complex task, you can't see the traceback so hard to debug

asksol: a simpler task succeeds: http://pastie.org/829783

asksol: Am I wasting my time with this test script?

asksol: more importantly, am I wasting your time?

there's a bug in your task, I guess

try to run it manually outside the pool (Task.run(...))