This roadmap ensures that the tasks needed to add a new source system (SRC) to the BI Data Lake are performed. The Source and BI Data Lake areas(Staging and History) will be supplied with ETL- and database code.
The tasks of the ‘Source to BI Data Lake – Roadmap’ are schematically displayed as follows:

This roadmap describes the goal, the end result and the work which has to be performed for each task.

The roadmap can be read as a ‘cookbook’ with video’s which are on the page next to this.

SRC-CON; Define the source configuration for the Powerpack generator.
Possible sources: Oracle, MySQL, SQLServer ,PostgreSQL, Snowflake, CSV, JSON and XML.
Define ini-file :(Initialisation properties) and config-file (Configuration for the source).

SMM-PPK; Define the Metadata for the source areas based on the source structure with the Source Metadata Manager (SMM).
• Fill the SMM tables with default metadata based on the source system SMM-PPK (part1).
• Modify the SMM tables SMM-PPK (part2) .
SMM-CHK; Check the consistency of the SMM (Source Metadata) definitions.
The consistency is checked before the productivity boosters are using the SMM metedata for generating the database DDL and python ETL software.

STG-DLL; Generate DDL for the Staging-area
Generate the DDL script for implementing the database objects (table) for the Staging area in the database.
STG-ETL; Generate ETL for the Staging-area
Generate the ETL scripts for loading the source data in the Staging area in the database.
STG-DEP; Deploy the DDL and ETL for the Staging-area.
Implement the generated database objects in your Cloud- or On-Premise database.
STG-RUN; Run ETL for the Staging-area.
Execute the ETL to load the load from the source into the Staging area tables included logging and error handling.

HIS-DLL; Generate DDL for the History-area
Generate the DDL script for implementing the database objects (table and sequence) for the History-area in the database.
HIS-ETL; Generate ETL for the History-area
Generate the ETL scripts for loading the source data in the History-area in the database.
• Historize records based on attribute-level changes.
• Incermental update data changes.
• Logical delete records;
HIS-DEP; Deploy the DDL  for the History-area.
Implement the generated database objects in your Cloud- or On-Premise database.
HIS-RUN; Run ETL for the History-area.
Execute the ETL to load the load from the Staging-area into the History-area tables  included logging and error handling.