Data Science Dojo delivers data science education, consulting, and technical services to harvest the power of data.

Trademarks: This software listing is packaged by Data Science Dojo. The respective trademarks mentioned in the offering are owned by the respective companies, and their use of them does not imply any affiliation or endorsement.

License: https://github.com/airbytehq/airbyte/blob/master/LICENSE

Airbyte is an attractive data-ingestion ELT tool because of its open-source, no/low-code, and community-driven nature. It decreases the amount of custom engineering required to load and store data. Airbyte pitches a no-code/low-code solution to reduce the amount of development required. The target audience seems to be businesses with small teams, lacking the time and money for skilled engineers and complicated pipelines. It is designed to seamlessly extract data from a source and load it into a data warehouse (along with many different options). Why use Airbyte for that? Because developing and managing data pipelines rely on costly engineering knowledge. A tool like Airflow requires writing custom python code to load and store data, giving a higher amount of customizability but things can get complex quickly. Airbyte's no-code approach tries to decrease the amount of custom code required. The more you avoid building and maintaining custom systems for data ingestion and transformation, the greater the amount of time you can spend on creating insights for your actual business. After all, managing data pipelines are only used to get actual value coming from the insights derived from the data.

• Data Analysts
• Data Engineers
• Database Managers
• And anyone who is looking to make ELT pipelines simple, secure, and extensible.

Ubuntu 20.04 LTS preconfigured with Airbyte, so you don't have to worry about setting up the environment.
Airbyte is on a mission to make data integration pipelines a commodity.

• High extensibility: Use existing connectors to your needs or build a new one with ease.
• Customization: Entirely customizable, start with raw data or from some suggestion of normalized data.
• Full-grade scheduler: Automate your replications with the frequency you need.
• Real-time monitoring: Logs all the errors in full detail to help you understand better.
• Incremental updates: Automated replications are based on incremental updates to reduce your data transfer costs.
• Manual full refresh: Re-syncs all your data to start again whenever you want.
• Debugging: Debug and Modify pipelines as you see fit, without waiting.

• Minimum memory: 8 GB
• Minimum vCPU: 2 vCPUs
• Operating System: Ubuntu 20.04

Create a VM with this image and you're good to go!

The default port Airbyte listens to is 8000. You can access the Airbyte's web interface at http://yourip:8000 using the credentials:

• username: airbyte
• password: password

You should update these values by changing BASIC_AUTH_USERNAME and BASIC_AUTH_PASSWORD in your local .env file.