Data Engineering on Azure

Azure Platform provides large number of services which allow to address business challenges using data solutions. Selection of the right services and their combination is a key factor in building an effective solution

Book our workshop to get familiar with Azure Data stack and understand how to efficiently use available services. Understand Azure data solutions development lifecycle. This workshop is dedicated for Data Engineers who want to understand basic and more advanced usage scenarios and how to combine together: Azure Data Lake Storage Gen2, Azure SQL Database, Azure Synapse Analytics, Azure Databricks, Azure Data Factory, Azure DevOps. During workshops you will learn how to integrate data services with Azure Key Vault and also understand security concepts of particular data services to integrate your data in secure way.

Workshop agenda:

• Day 1: Data concepts, Azure Cloud Basics, Azure Storage: Understand data platform concepts, get familiar with Azure Cloud and Azure Portal, setup your Azure environment, understand Azure Data Lake Storage Gen2 and tools like AzCopy, Azure Storage Explorer
• Day 2: Database lifecycle and data orchestration in Azure: Understand Azure SQL service basics, get familiar with tools like SSMS, SSDT, ADS. Go through Azure SQL Database development lifecycle. Understand Azure Data Factory and build first cloud data pipeline.
• Day 3: Generic ETL and Azure Synapse Analytics: Understand how to build more advanced ADF pipelines i.e. data-driven ETL process. Learn Azure Synapse Analytics platform, understand the basics, concepts and usage scenarios for SQL Serverless Pool, SQL Dedicated Pool, Apache Spark Pool and build ETL pipeline within Synapse.
• Day 4: Azure Databricks: Get familiar with Azure Databricks platform. Understand platform architecture, concepts and features. Learn how to integrate Azure Databricks with Azure Data Lake, Azure SQL Database, Azure Synapse Analytics and Azure Key Vault. See how to build generic code and orchestrate it through Azure Data Factory
• Day 5: DevOps and Do it yourself Get familiar with DevOps concepts and Azure DevOps service. Understand the platform features and possibilities. Learn how to integrate your code with Azure DevOps GIT repository for Azure SQL Databases, Azure Data Factory and Azure Databricks. Get familiar with GIT Flow concept. In the second of part the day, participants with build from scratch end-to-end data solution using all known services.

Maximum number of participants: 10