Azure Data Factory Pipeline: 1-Week Assessment

Azure Data Factory is Azure’s ETL/ELT cloud service for serverless data pipelines with the purpose of data integration and/or data transformation. Azure Data Factory data pipelines and processes can become very complex over time. This complexity of the pipelines can cause difficulties in understanding the flow of the process and, therefore, become hard to troubleshoot in case of failure or unexpected behavior. Moreover, unnecessarily complex data pipelines can and will create extra costs. Therefore, we can help our customers by simplifying and optimizing Azure Data Factory, which will lead to a reduction in monthly Azure costs while maintaining at least the same productivity. We offer to help companies that are already using Azure Data Factory pipelines and are struggling to understand the current Azure Data Factory pipelines, or for companies that want to optimize and reduce the time and cost for Azure Data Factory pipelines run. We will start creating the full picture of your Azure Data Factory pipelines by first understanding it ourselves and then explaining it to you via a presentation. Finally, we will present the plan for your Azure Data Factory pipelines and processes by stating the clear value to you in terms of simplicity, optimization, reduced overall pipeline run time and cost.

What we offer: • An initial assessment of the current architecture, with an emphasis of the aspects that could use improvement and suggestions to do so. • Main focus would be on achieving the same or better results, using fewer resources, less runtime and lower cost. • The results of the assessment will be communicated via a presentation (on-site if feasible, online otherwise). • Discussion of what aspects are the most important to improve based on the business and technical needs.

Deliverables: • Proposed optimized pipeline architecture in form of presentation and diagrams will be made available to you.

Outcome: • Explanation of improvement and The What's, The Why's and The How's including clear understanding of cost reduction.

Next Steps: • After the assessment, if you are interested in implantation of the proposed solution can be less or more than the proposed three weeks, depending on what exactly needs to be done (exact time will be decided after the assessment and discussion) • If the required changes are too substantial, re-creating the architecture from scratch might be a better option.