Data Science Dojo delivers data science education, consulting, and technical services to harvest the power of data.

Trademarks: This software listing is packaged by Data Science Dojo. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

About the offer:

Celery is an open-source tool used to implement task queues in real time and is based on distributed message passing. Celery is used in conjunction with a message broker. A common message broker that is used with celery is Redis which is pre-configured in this offer. This offer also provides Flower, a web-based tool for monitoring and administrating Celery tasks. Our instance allows the user to get some observability of the running Celery background task such as how the task is performed, how long it takes to run a task, spot failures, and debug their behavior with the help of the Flower dashboard.

Who benefits from this offer:

Following can benefit from Celery:

Backend Developers
Data Engineers
Software Engineers

What is included in this offer:

Flower(web-based tool) to observe flower task progress and history
Redis setup is pre-configured at port 6379 in this offer as Redis is used to store results coming off the celery queues which are then retrieved by consumers of the queue.
A sample celery task is included in this offer which reverses the text provided and is located at /home/tasks.py .You can monitor the task at http://yourip:5555 after typing the following commands:
- sudo docker start redis
- cd ..
- celery -A tasks worker --loglevel=info
- celery -A tasks flower --loglevel=info

Technical Specifications:

Minimum memory: 8GB RAM
Minimum vCPU: 2 vCPUs
Operating System: Ubuntu 20.04

Celery Supports:

Message Transports

RabbitMQ
Redis
Amazon SQS

Concurrency

Prefork
gevent
single threaded
Eventlet

Result Stores

AMQP, Redis
memcached
SQLAlchemy, Django ORM
Apache Cassandra, IronCache, Elasticsearch

Serialization

pickle, json, yaml, msgpack.
zlib, bzip2 compression.
Cryptographic message signing

Integration with frameworks:

Celery is easy to integrate with web frameworks. Some of the frameworks it supports are:

Django
Pyramid
Pylons
Flask
web2py
Tornado
Tryton

Commands to connect Celery Tasks with flower:

sudo docker start redis
cd ..
celery -A 'celery task name' worker --loglevel=info
celery -A 'celery task name' flower --loglevel=info

The default port Flower listens to is 5555. You can access the web interface at http://yourip:5555

Celery with Flower UI and Redis packaged by Data Science DojoData Science Dojo

Celery with Flower UI and Redis packaged by Data Science Dojo

Data Science Dojo

Celery with Flower UI and Redis packaged by Data Science Dojo

Data Science Dojo

Celery with Flower UI and Redis packaged by Data Science Dojo
Data Science Dojo