Tiger Data Fabric: Accelerator for Implementing Data Lake & Data Fabric on Microsoft Azure

Tiger Data Fabric is developed by following various industry best practices and using proven Azure PaaS & Open-Sourced Big Data services. Some of the key features include:

1. Agility with Self Service: Cuts down the time it takes for setting up new Data Pipelines from Days & Weeks to Few Minutes. Leverage different interfaces like Chatbot, Automation Scripts, REST APIS and a modern Web UI for managing the entire data platform and configure new data pipelines.

2. Medallion Lakehouse Architecture: Follows Data Engineering Best Practices & based on Medallion Lakehouse Architecture by organizing the data in Bronze, Silver & Gold layers.

3. Scalable Cloud Architecture: Highly Scalable & Service Oriented Architecture, developed using different Azure PaaS & Open-Source Big Data Technologies.

4. Transformation Capabilities: Capabilities to perform various data Transformation Operations like data cleansing, validation, quarantine, auto-profile & target merge – in a highly configuration driven manner. Integrated DBT Module for implementing governed transformation pipelines.

5. DE Virtual Assistant: An integrated virtual assistant (powered by Microsoft Bot framework and LUIS) that provides capabilities for setting up ingestion pipelines, tracking jobs, evaluating data quality etc. – all over a simple chat interface.

6. Knowledge Graph & Intelligent Search: Integrated with Knowledge graph & Search Index to provide intelligent Data Catalog capabilities (for both metadata and lineage).

7. Data Quality: Great Expectations based Data Testing framework to validate onboarded datasets for consistency and user defined rules.

8. Additional Governance Capabilities: Powerful framework for logging, notification & alert, and azure cost monitoring features.

9. DevOps: Powerful Infrastructure as Code capabilities with single click deployment via CI/CD pipelines.

Technologies Used: Azure Data Factory, Azure Databricks, Azure SQL Server, Azure Data Lake Storage Gen2, Elastic Search, Azure Key vault, Azure DevOps