Source Data Ingestion

First step to build Cloud Data Analytics or Modern Data Platform in any way is usually to gather all the date in one place. As Sii we propose to use proven approach that was implemented and tested with several customers. Solution is fully opened, meaning you have access to every service, code or deployment scripts without any obfuscation or “black-box” elements. What’s more it is created on top of native Azure components, so any person with programming knowledge can maintain and extend it.

Some services that we use:

• Azure Key Vault, • Azure Storage \ Data Lake, • Azure Data Factory \ Azure Synapse Pipelines, • Azure Synapse Workspaces, • Azure Synapse Spark Pool, • Azure SQL database, • and others when needed.

Solution is also implemented using the benefits of DevOps approach meaning full CI\CD pipelines and Infrastructure as a Code elements.

This solutions comes with:

• Data source analysis and documentation, • Configuring existing connectors or development of new connectors to data sources, • Creating Azure Data Factory or Azure Synapse pipelines to load raw data into the cloud, • Fully documented solution and environment, • Setup of DevOps pipeline including Infrastructure as a Code elements, Pull Request policies, automatic tests or deployments, • Additional services like Azure infrastructure audits and setup, security setup etc.