Constructing a contemporary data warehouse that spans multiple databases, with complete ETL pipelines, structured data models, and robust analytics tools.
This project involves:
- Data Architecture: Designing a Modern Data Warehouse Using Medallion Architecture Bronze, Silver, and Gold layers.
- ETL Pipelines: Extracting, transforming, and loading data from source systems into the warehouse.
- Data Modeling: Developing fact and dimension tables optimized for analytical queries.
- Analytics & Reporting: Creating SQL-based reports and dashboards for actionable insights.
The architecture for this project follows Medallion Architecture Bronze, Silver, and Gold layers:
- Bronze Layer: Stores raw data as-is from the source systems. Data is ingested from CSV Files into the database.
- Silver Layer: This layer includes data cleansing, standardization, and normalization processes to prepare data for analysis.
- Gold Layer: Houses business-ready data modeled into a star schema required for reporting and analytics.
If you notice any problems with running this, please open an issue here.
- We welcome contributions through forking the repository to address issues or implement new features.
- Please include appropriate test coverage for all submitted code.
- After submission, your pull request will undergo review and testing before being merged into the main codebase.