许å¤Djangoåºç¨éè¦æ§è¡å¼æ¥ä»»å¡, 以便ä¸è½è¯¯http requestçæ§è¡. æ们ä¹å¯ä»¥éæ©è®¸å¤æ¹æ³æ¥å®æå¼æ¥ä»»å¡, 使ç¨Celeryæ¯ä¸ä¸ªæ¯è¾å¥½çéæ©, å 为Celery
æç大éç社åºæ¯æ, è½å¤å®ç¾çæ©å±, åDjangoç»åçä¹å¾å¥½. Celeryä¸ä»
è½å¨Djangoä¸ä½¿ç¨, è¿è½å¨å
¶ä»å°æ¹è¢«å¤§éç使ç¨. å æ¤ä¸æ¦å¦ä¼ä½¿ç¨Celery, æ
们å¯ä»¥å¾æ¹ä¾¿çå¨å
¶ä»é¡¹ç®ä¸ä½¿ç¨å®.
1. Celeryçæ¬
æ¬ç¯åæ主è¦é对Celery 3.0.x. æ©æçæ¬çCeleryå¯è½æç»å¾®çå·®å«.
2. Celeryä»ç»
Celeryç主è¦ç¨å¤æ¯æ§è¡å¼æ¥ä»»å¡, å¯ä»¥éæ©å»¶ææå®æ¶æ§è¡åè½. 为ä»ä¹éè¦æ§è¡å¼æ¥ä»»å¡å¢?
第ä¸, å设ç¨æ·æ£åèµ·ä¸ä¸ªrequest, 并çå¾
requestå®æåè¿å. å¨è¿ä¸requeståé¢çviewåè½ä¸, æ们å¯è½éè¦æ§è¡ä¸æ®µè±è´¹å¾é¿æ¶é´çç¨åºä»»å¡, è¿ä¸æ¶é´
å¯è½è¿è¿å¤§äºç¨æ·è½å¿åçèå´. å½è¿ä¸ä»»å¡å¹¶ä¸éè¦ç«å»æ§è¡æ¶, æ们便å¯ä»¥ä½¿ç¨Celeryå¨åå°æ§è¡, èä¸å½±åç¨æ·æµè§ç½é¡µ. å½æä»»å¡éè¦è®¿é®è¿ç¨æå¡å¨å®
ææ¶, æ们å¾å¾é½æ æ³ç¡®å®éè¦è±è´¹çæ¶é´.
第äºåæ¯å®ææ§è¡æäºä»»å¡. æ¯å¦æ¯å°æ¶éè¦æ£æ¥ä¸ä¸å¤©æ°é¢æ¥, ç¶åå°æ°æ®å¨åå°æ°æ®åºä¸. æ们å¯ä»¥ç¼åè¿ä¸ä»»å¡, ç¶å让Celeryæ¯å°æ¶æ§è¡ä¸æ¬¡. è¿æ ·æ们
çwebåºç¨ä¾¿è½è·åææ°ç天æ°é¢æ¥ä¿¡æ¯.
æ们è¿éæ讲çä»»å¡task, å°±æ¯ä¸ä¸ªPythonåè½(function). å®ææ§è¡ä¸ä¸ªä»»å¡å¯ä»¥è¢«è®¤ä¸ºæ¯å»¶æ¶æ§è¡è¯¥åè½. æ们å¯ä»¥ä½¿ç¨Celery延è¿5åéè°ç¨function
task1, å¹¶ä¼ å
¥åæ°(1, 2, 3). æè
æ们ä¹å¯ä»¥æ¯å¤©åå¤è¿è¡è¯¥function.
æ们ååäºå°Celeryæ¾å
¥é¡¹ç®ä¸, 便äºtask访é®ç»ä¸æ°æ®åºåDjango设置.
å½taskåå¤è¿è¡æ¶, Celeryä¼å°å
¶æ¾å
¥åéqueueä¸. queueä¸å¨åçå¯ä»¥è¿è¡çtaskçlist. æ们å¯ä»¥ä½¿ç¨å¤ä¸ªqueue, ä½ä¸ºäºç®å, è¿éæ们åªä½¿ç¨ä¸ä¸ª.
å°ä»»å¡taskæ¾å
¥queueå°±åå å
¥todo listä¸æ ·. 为äºä½¿taskè¿è¡, æ们è¿éè¦å¨å
¶ä»çº¿ç¨ä¸è¿è¡çè¦å·¥worker. workerå®æ¶è§å¯ç代è¿è¡çtask, 并éä¸è¿è¡è¿
äºtask. ä½ å¯ä»¥ä½¿ç¨å¤ä¸ªworker, é常ä»ä»¬ä½äºä¸åæå¡å¨ä¸. åæ ·ä¸ºäºç®åèµ·è§, æ们è¿åªæ¯ç¨ä¸ä¸ªworker.
æ们ç¨åä¼è®¨è®ºqueue, workeråå¦å¤ä¸ä¸ªååéè¦çè¿ç¨, æ¥ä¸æ¥æ们æ¥å¨å¨æ:
3. å®è£
Celery
æ们å¯ä»¥ä½¿ç¨pipå¨vietualenvä¸å®è£
:
pip install django-celery
4. Django设置
æ们ææ¶ä½¿ç¨django runserveræ¥å¯å¨celery. èCelery代ç人(broker), æ们使ç¨Django database broker implementation. ç°å¨æ们åªéè¦ç¥éCelery
éè¦broker, 使ç¨djangoèªèº«ä¾¿å¯ä»¥å
å½broker. (ä½å¨é¨ç½²æ¶, æ们æ好使ç¨æ´ç¨³å®åé«æçbroker, ä¾å¦Redis.)
å¨settings.pyä¸:
import djcelery
djcelery.setup_loader()
BROKER_URL = 'django://'
...
INSTALLED_APPS = (
...
'djcelery',
'kombu.transport.django',
...
)
第ä¸äºé¡¹æ¯å¿
é¡»ç, 第ä¸é¡¹ååè¯Celery使ç¨Django项ç®ä½ä¸ºbroker.
å¨INSTALLED_APPSä¸æ·»å çdjceleryæ¯å¿
é¡»ç. kombu.transport.djangoåæ¯åºäºDjangoçbroker
æåå建Celeryæéçæ°æ®è¡¨, å¦æ使ç¨Southä½ä¸ºæ°æ®è¿ç§»å·¥å
·, åè¿è¡:
python manage.py migrate
å¦åè¿è¡: (Django 1.6æDjango 1.7é½å¯ä»¥)
python manage.py syncdb
5. å建ä¸ä¸ªtask
æ£å¦åé¢æ说ç, ä¸ä¸ªtaskå°±æ¯ä¸ä¸ªPyhton function. ä½Celeryéè¦ç¥éè¿ä¸functionæ¯task, å æ¤æ们å¯ä»¥ä½¿ç¨celeryèªå¸¦çè£
饰å¨decorator: @task. å¨
django appç®å½ä¸å建taske.py:
from celery import task
@task()
def add(x, y):
return x + y
å½settings.pyä¸çdjcelery.setup_loader()è¿è¡æ¶, Celery便ä¼æ¥çææINSTALLED_APPSä¸appç®å½ä¸çtasks.pyæ件, æ¾å°æ 记为taskçfunction, 并
å°å®ä»¬æ³¨å为celery task.
å°functionæ 注为task并ä¸ä¼å¦¨ç¢ä»ä»¬çæ£å¸¸æ§è¡. ä½ è¿æ¯å¯ä»¥åå¹³æ¶é£æ ·è°ç¨å®: z = add(1, 2).
6. æ§è¡task
让æ们以ä¸ä¸ªç®åçä¾åä½ä¸ºå¼å§. ä¾å¦æ们å¸æå¨ç¨æ·ååºrequeståå¼æ¥æ§è¡è¯¥task, 马ä¸è¿åresponse, ä»èä¸é»å¡è¯¥request, 使ç¨æ·æä¸ä¸ªæµç
ç访é®
è¿ç¨. é£ä¹, æ们å¯ä»¥ä½¿ç¨.delay, ä¾å¦å¨å¨views.pyçä¸ä¸ªviewä¸:
from myapp.tasks import add
...
add.delay(2, 2)
...
Celeryä¼å°taskå å
¥å°queueä¸, 并马ä¸è¿å. èå¨ä¸æå¾
å½çworkerçå°è¯¥taskå, 便ä¼æç
§è®¾å®æ§è¡å®, 并å°ä»ä»queueä¸ç§»é¤. èworkeråä¼æ§è¡ä»¥ä¸ä»£
ç :
import myapp.tasks.add
myapp.tasks.add(2, 2)
7. å
³äºimport
è¿ééè¦æ³¨æçæ¯, å¨impprt taskæ¶, éè¦ä¿æä¸è´. å 为å¨æ§è¡djcelery.setup_loader()æ¶, taskæ¯ä»¥INSTALLED_APPSä¸çappå,
å .tasks.function_name注åç, å¦ææ们ç±äºpython pathä¸åè使ç¨ä¸åçå¼ç¨æ¹å¼æ¶(ä¾å¦å¨tasks.pyä¸ä½¿ç¨from myproject.myapp.tasks import
addå½¢å¼), Celeryå°æ æ³å¾ç¥è¿æ¯åä¸task, å æ¤å¯è½ä¼å¼èµ·å¥æªçbug.
8. æµè¯
a. å¯å¨worker
æ£å¦ä¹å说å°ç, æ们éè¦workeræ¥æ§è¡task. 以ä¸æ¯å¨å¼åç¯å¢ä¸çå¦ä½å¯å¨worker:
é¦å
å¯å¨terminal, å¦åå¼ådjango项ç®ä¸æ ·, æ¿æ´»virtualenv, åæ¢å°django项ç®ç®å½. ç¶åå¯å¨djangoèªå¸¦webæå¡å¨: python manage.py runserver.
ç¶åå¯å¨worker:
python manage.py celery worker --loglevel=info
æ¤æ¶, workerå°ä¼å¨è¯¥terminalä¸è¿è¡, 并æ¾ç¤ºè¾åºç»æ.
b. å¯å¨task
æå¼æ°çterminal, æ¿æ´»virtualenv, 并åæ¢å°django项ç®ç®å½:
$ python manage.py shell
>>> from myapp.tasks import add
>>> add.delay(2, 2)
æ¤æ¶, ä½ å¯ä»¥å¨workerçªå£ä¸çå°workeræ§è¡è¯¥task:
[2014-10-07 08:47:08,076: INFO/MainProcess] Got task from broker: myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc]
[2014-10-07 08:47:08,299: INFO/MainProcess] Task myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc] succeeded in 0.183349132538s: 4
9. å¦ä¸ä¸ªä¾å
ä¸é¢æ们æ¥çä¸ä¸ªæ´ä¸ºçå®çä¾å, å¨views.pyåtasks.pyä¸:
# views.py
from myapp.tasks import do_something_with_form_data
def view(request):
form = SomeForm(request.POST)
if form.is_valid():
data = form.cleaned_data
# Schedule a task to process the data later
do_something_with_form_data.delay(data)
return render_to_response(...)
# tasks.py
@task
def do_something_with_form_data(data):
call_slow_web_service(data['user'], data['text'], ...)
10. è°è¯
ç±äºCeleryçè¿è¡éè¦å¯å¨å¤ä¸ªé¨ä»¶, æ们å¯è½ä¼æ¼æä¸ä¸¤ä¸ª. æ以æ们建议:
使ç¨æç®åç设置
使ç¨python debugåloggingåè½æ¾ç¤ºå½åçè¿ç¨
11. Eager模å¼
å¦æå¨settings.py设置:
CELERY_ALWAYS_EAGER = True
é£ä¹Celery便以eager模å¼è¿è¡, åtask便ä¸éè¦å delayè¿è¡:
# è¥å¯ç¨eager模å¼, å以ä¸ä¸¤è¡ä»£ç ç¸å
add.delay(2, 2)
add(2, 2)
12. æ¥çqueue
å 为æ们使ç¨äºdjangoä½ä¸ºbroker, queueå¨åå¨djangoçæ°æ®åºä¸. è¿å°±æå³çæ们å¯ä»¥éè¿django adminæ¥ç该queue:
# admin.py
from django.contrib import admin
from kombu.transport.django import models as kombu_models
admin.site.register(kombu_models.Message)
13. æ£æ¥ç»æ
æ¯æ¬¡è¿è¡å¼æ¥taskå, Celeryé½ä¼è¿åAsyncResult对象ä½ä¸ºç»æ. ä½ å¯ä»¥å°å
¶ä¿å, ç¶åå¨å°æ¥æ¥ç该taskæ¯å¦è¿è¡æååè¿åç»æ:
# views.py
result = add.delay(2, 2)
...
if result.ready():
print "Task has run"
if result.successful():
print "Result was: %s" % result.result
else:
if isinstance(result.result, Exception):
print "Task failed due to raising an exception"
raise result.result
else:
print "Task failed without raising exception"
else:
print "Task has not yet run"
14. å®æä»»å¡
è¿æä¸ç§Celeryç常ç¨æ¨¡å¼ä¾¿æ¯æ§è¡å®æä»»å¡. æ§è¡å®æä»»å¡æ¶, Celeryä¼éè¿celerybeatè¿ç¨æ¥å®æ. Celerybeatä¼ä¿æè¿è¡, ä¸æ¦å°äºæä¸å®æä»»å¡éè¦æ§
è¡æ¶, Celerybeat便å°å
¶å å
¥å°queueä¸. ä¸åworkerè¿ç¨, Celerybeatåªæéè¦ä¸ä¸ªå³å¯.
å¯å¨Celerybeat:
python manage.py celery beat
使Celeryè¿è¡å®æä»»å¡çæ¹å¼æå¾å¤ç§, æ们å
ç第ä¸ç§, å°å®æä»»å¡å¨åå¨djangoæ°æ®åºä¸. å³ä½¿æ¯å¨djangoåceleryé½è¿è¡çç¶æ, è¿ä¸æ¹å¼ä¹å¯ä»¥è®©æ们
æ¹ä¾¿çä¿®æ¹å®æä»»å¡. æ们åªéè¦è®¾ç½®settings.pyä¸çä¸é¡¹ä¾¿è½å¼å¯è¿ä¸æ¹å¼:
# settings.py
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
温馨提示:答案为网友推荐,仅供参考