ETL GINI- Teradata Integration Suite

Abstract

This project is developed specially for automating the entire current ETL process at NCR. The entire process takes a long time. This product dramatically reduces the time by automatically generating various things, like Scripts Generation, Mapping, SQL generation, and Report generation. This documentation describes each and every aspect of the application from analysis to the design and then to the implementation phases. Compiling mountains of raw data that has been created by modem and multinational organizations is a task that is best described as difficult. As the organizations and companies evolve, they tend to acquire different applications, operating systems, hardware platfonns, databases as well as many other fonns of software. With all of these applications usually spread across several departments, the data tends to become increasingly isolated as well as unusable due to wastage. When the data is moved to an Enterprise Data Warehouse (EDW), the developers need to extract the disparate data, transfonn and load it into the EDW. The process as a whole was very lengthy and full of errors not to mention painful. This was mostly down to complex data models which the EDW had. In addition, the configuration management was also difficult to maintain. Even after the ETL process, the meta data always remained a secret. Moreover, during and after successfully achieving the CMM level 5, the need for institutionalizing the data warehouse development became increasingly necessary. The prime motive of the organizations was to make sure that more time is spent in providing specialized industry specific knowledge, in the area of Business Intelligence and post deployment support in the usage of data instead of spending more time in the ETL phase. The usage is what makes a Data warehouse successful. Due to this very reason the creation of a state of the art integration tool was required. So that this tool helps cut down the production time in the data mapping phase. It is known as the ETL Gini. It produces the Source Transformation Matrix (SMX) and standardized BTEQ, Fast Load, Fast Export and Multi Load scripts. ETL Gini- Teradata Integration