https://store-images.s-microsoft.com/image/apps.23773.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.e7808266-83b5-43a5-9cbb-2dd81c725875.535a240a-3655-43f1-86b6-27b0a7e6138b

Celery with Flower UI and Redis packaged by Data Science Dojo

Data Science Dojo

Celery with Flower UI and Redis packaged by Data Science Dojo

Data Science Dojo

Setup your Celery tasks with Redis and monitor them through Flower dashboard.

Data Science Dojo delivers data science education, consulting, and technical services to harvest the power of data.

Trademarks: This software listing is packaged by Data Science Dojo. The respective trademarks mentioned in the offering are owned by the respective companies, and use of them does not imply any affiliation or endorsement.

About the offer:

Celery is an open-source tool used to implement task queues in real time and is based on distributed message passing. Celery is used in conjunction with a message broker. A common message broker that is used with celery is Redis which is pre-configured in this offer. This offer also provides Flower, a web-based tool for monitoring and administrating Celery tasks. Our instance allows the user to get some observability of the running Celery background task such as how the task is performed, how long it takes to run a task, spot failures, and debug their behavior with the help of the Flower dashboard.

Who benefits from this offer:

Following can benefit from Celery:

  • Backend Developers
  • Data Engineers
  • Software Engineers

What is included in this offer:
  • Flower(web-based tool) to observe flower task progress and history
  • Redis setup is pre-configured at port 6379 in this offer as Redis is used to store results coming off the celery queues which are then retrieved by consumers of the queue.
  • A sample celery task is included in this offer which reverses the text provided and is located at /home/tasks.py .You can monitor the task at http://yourip:5555 after typing the following commands:
    • sudo docker start redis
    • cd ..
    • celery -A tasks worker --loglevel=info
    • celery -A tasks flower --loglevel=info
Technical Specifications:
  • Minimum memory: 8GB RAM
  • Minimum vCPU: 2 vCPUs
  • Operating System: Ubuntu 20.04
Celery Supports:
  • Message Transports

    • RabbitMQ
    • Redis
    • Amazon SQS
  • Concurrency

    • Prefork
    • gevent
    • single threaded
    • Eventlet
  • Result Stores

    • AMQP, Redis
    • memcached
    • SQLAlchemy, Django ORM
    • Apache Cassandra, IronCache, Elasticsearch
  • Serialization

    • pickle, json, yaml, msgpack.
    • zlib, bzip2 compression.
    • Cryptographic message signing
Integration with frameworks:

Celery is easy to integrate with web frameworks. Some of the frameworks it supports are:

  • Django
  • Pyramid
  • Pylons
  • Flask
  • web2py
  • Tornado
  • Tryton
Commands to connect Celery Tasks with flower:
  • sudo docker start redis
  • cd ..
  • celery -A 'celery task name' worker --loglevel=info
  • celery -A 'celery task name' flower --loglevel=info

The default port Flower listens to is 5555. You can access the web interface at http://yourip:5555

https://store-images.s-microsoft.com/image/apps.64796.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.ebf4fd15-1c5c-4e1c-bcb4-6c4132d72540
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.64796.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.ebf4fd15-1c5c-4e1c-bcb4-6c4132d72540
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.57873.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.15ff2de0-8612-4da0-817b-ad0630e9578b
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.63446.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.96b912b7-008c-4587-9051-1bbcdc84db74
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.23374.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.0db9864a-50d1-4898-bcec-759ce1cece24
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.61723.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.3d7f11d8-3daa-4231-a821-92e3f08ace48
/staticstorage/8165fe0/assets/videoOverlay_7299e00c2e43a32cf9fa.png
https://store-images.s-microsoft.com/image/apps.2453.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.14365870-f321-4990-9001-21ce5a32165c
https://store-images.s-microsoft.com/image/apps.39577.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.7ad9fddf-62ca-441d-9871-9d29422ccb4f
https://store-images.s-microsoft.com/image/apps.4817.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.50c837c7-0583-486a-8288-fce06d030d4d
https://store-images.s-microsoft.com/image/apps.737.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.51b2f03d-ce08-4914-a83b-cfa1ec877cc8
https://store-images.s-microsoft.com/image/apps.46004.8a86ffd8-c28e-4f25-8bc5-68af77e81ecd.6f16b94b-f890-4f22-ad40-9bebca627b82.1dd3a887-2d01-4b87-8028-4ba7e73bec93