19 Aug 2020

CS Company has completed a large-scale project of the corporate data warehouse CS::BM implementation in OTP Bank. 

One of the top priorities of OTP Bank's IT strategy was to replace the existing data warehouse with an industrial solution. CS Company was chosen as a provider, as the bank has been cooperating with the company since 2009.

GOALS AND TASKS 

• Implement a modern corporate data warehouse as one reliable data source in the bank: with a domain-specific data warehouse data model covering as many entities as possible, and data quality control. 
• Migrate the functionality and data of the two existing corporate data warehouses into a single modernized corporate data warehouse. Transfer of the existing bank's warehouse data marts.
• Implement data loading at T+1 / (T = transaction date).
• Increase the amount of data/entities in the warehouse required by business users.
• Get a fault-tolerant managed solution.

FEATURES OF IMPLEMENTATION

Developed:

• Mechanisms for managing additional data loading.
• Data quality control module.
• Solutions for providing data for group reporting.

Completed:

• Improvements regarding the control of the data loading process and data marts calculation.
• Refactoring of the bank warehouse code that was used earlier.

The CS::BM solution is built on the traditional large DWH methodology proposed by Bill Inmon. The methodology is based on a centralized storage of all corporate data, on their basis further data marts are formed. 

DBMS – Oracle 12C. 
ETL/ELT – Oracle Data Integrator (ODI) 12C.
Data mart visualization tool – Oracle Business Intelligence (BI) 12C.

Sergey Rachkov, Business Analyst of the CS Company:
“DWH in OTP Bank is filled from more than a dozen sources. Each of the systems has its own data loading regulations. The analysis of links between downloads is performed at the data warehouse core level in order to correctly approach the implementation of data marts.
Daily the data from 700 tables are uploaded and more than 50 different data marts are formed, which are used directly by users via Oracle BI, and used for integration with systems – data receivers. About eight systems consume data calculated in the data warehouse. These systems are built on various databases, use different approaches to loading data. All this necessitated the development of completely different intersystem interfaces, which formed the overall architecture of the solution”.

Oleg Rozum, Head of DWH Department Development, Manager of the Project, OTP Bank: 
“The Bank's project team and CS Ltd company had ambitious tasks:
consolidate analytical reporting at the single DWH solution level using the optimal amount of resources;

• update the hardware and software of the DWH solution;
• migrate and modernize data marts from old DWH solutions, as well as transfer labor-intensive analytical reporting from Master systems to the new DWH;
• document the new DWH solution;
• eliminate architectural and logical "gaps" of old solutions;
• ensure the security of data storage and processing;
• provide efficient data storage;
• implement a data quality control model;
• initiate a reporting approach based on BI solution – «Business Self Service».

Thanks to joint efforts, mutual understanding, result-oriented cooperation, the project was successfully completed”.

It was used Knowledge Modules designed by CS Company during the implementation, for working in Oracle Data Integrator:

• Loading Knowledge Modules (LKM).
  • For unloading from systems to Oracle DB.
  • For unloading from systems to MS-SQL.
  • For unloading files from the file system, ftp.
• Integration Knowledge Modules (IKM).
  • For downloading to DWH.
  • For data marts forming.
  • For unloading prepared information to other information systems.

ADVANTAGES OF USING CS::BM in OTP BANK

• Bank has received the solution that can be supported.
• CS::BM data smarts are calculated faster than in the previous solution, which was critical for the bank.
• Provided timely submission of group reports.
• Formed the correct process of working with DWH – support, local development, vendor.
• Expandability – if necessary, various new data sources can be connected to the system.
• A single vendor company for key systems used in IT architecture, which allows you to implement changes in DWH in advance after changes implemented in core systems.
• Transfer of several resource-intensive tasks from source systems to the DWH side.