The LakeFusion Environment is the core of the platform and consists of two main layers:
Micro Frontends: The user interface components.
Microservices: The backend logic and processing components.
a. Micro Frontends
The LakeFusion Micro Frontends are built using front-end technologies like single-spa and React. This approach allows for modular development and deployment of individual portals. These portals include:
Root Portal:
Central access point for system-wide configurations, such as administrative tasks and global settings.
Main Portal:
Hosts core MDM functionalities like managing master data, deduplication, and data quality checks. It also includes tools for managing workflows and facilitating collaboration.
Utility Portal:
Provides supporting tools like user management, access logs, and troubleshooting documentation.
Each portal operates independently, ensuring better performance and flexibility.
b. Microservices
The backend logic resides in the Microservices layer, built with FastAPI and Python. Key services include:
Authentication Service:
Manages user authentication and authorization, integrated with Databricks SSO for secure access.
Databricks Service:
Acts as the communication bridge between LakeFusion and Databricks. This service handles:
Query execution on SQL Warehouses.
Orchestration of Databricks Workflows.
Access to Databricks resources such as Unity Catalog for data governance.
Middle Layer Service:
Connects the front end with backend services, facilitating API calls between the micro frontends and microservices.
AI/ML Service:
Powers advanced analytics with capabilities like:
Predictive analytics for identifying patterns in master data.
Deduplication algorithms to ensure data quality.
Machine learning models deployed for anomaly detection and forecasting.
c. Transactional Database
The backend relies on the LakeFusion Transactional Database, which supports MySQL, MS SQL, and PostgreSQL. It stores and manages:
Transactional data generated during user interactions with the platform.
Metadata related to workflows, datasets, and configurations.
Temporary and intermediate data used during analytics and processing.