Data Science Dojo delivers data science education, consulting, and technical services to harvest the power of data.

Trademarks: This software listing is packaged by Data Science Dojo. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

Celery is an open-source tool used to implement task queues in real time and is based on distributed message passing. Celery is used in conjunction with a message broker. A common message broker that is used with celery is Redis which is pre-configured in this offer. This offer also provides Flower, a web-based tool for monitoring and administrating Celery tasks. Our instance allows the user to get some observability of the running Celery background task such as how the task is performed, how long it takes to run a task, spot failures, and debug their behavior with the help of the Flower dashboard.

Following can benefit from Celery:

• Backend Developers
• Data Engineers
• Software Engineers

• Flower(web-based tool) to observe flower task progress and history
• Redis setup is pre-configured at port 6379 in this offer as Redis is used to store results coming off the celery queues which are then retrieved by consumers of the queue.
• A sample celery task is included in this offer which reverses the text provided and is located at /home/tasks.py .You can monitor the task at http://yourip:5555 after typing the following commands:
  • sudo docker start redis
  • cd ..
  • celery -A tasks worker --loglevel=info
  • celery -A tasks flower --loglevel=info

• Minimum memory: 8GB RAM
• Minimum vCPU: 2 vCPUs
• Operating System: Ubuntu 20.04

• Message Transports

• RabbitMQ
• Redis
• Amazon SQS

• Concurrency

• Prefork
• gevent
• single threaded
• Eventlet

• Result Stores

• AMQP, Redis
• memcached
• SQLAlchemy, Django ORM
• Apache Cassandra, IronCache, Elasticsearch

• Serialization

• pickle, json, yaml, msgpack.
• zlib, bzip2 compression.
• Cryptographic message signing

Celery is easy to integrate with web frameworks. Some of the frameworks it supports are:

• Django
• Pyramid
• Pylons
• Flask
• web2py
• Tornado
• Tryton

• sudo docker start redis
• cd ..
• celery -A 'celery task name' worker --loglevel=info
• celery -A 'celery task name' flower --loglevel=info

The default port Flower listens to is 5555. You can access the web interface at http://yourip:5555