Informatica Powercenter,ETL,ELT,Hadoop,Talend,Pentaho,SQL,PLSQL,Star Schema, Data Vault, Dimensional Modelling,Unix Shell Scripting, Data Migration
Aktualisiert am 01.08.2023
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 01.01.2024
Verfügbar zu: 100%
davon vor Ort: 100%
Python
Data Engineering
Azure
Oracle/SQL
Data Integration
Python
AWS
azure data factory
Databricks
Snowflake
ETL
Azure Data Engineer
Python Pandas
talend
apache spark
Deutsch
basic
Englisch
fluent

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

3 Monate
2023-02 - 2023-04

more projects

Microsoft Azure Azure Blobs REST API ...
Project 1:
Health, Safety, Security and the Environment (HSSE) Reporting Migration from
SQL Server to Snowflake Datawarehouse
This project involves extracting incident or cases data from Incident
Management Systems via REST APIs. These incident or cases are recorded at
various energy assets such as powerplants, units, weir, etc. The extracted data
then enriched with central asset master data and then loaded into report tables
for generating HSSE Reports in Tableau.


Project 2:Asset Management Reporting

This project involves extracting data from Azure Blob containers to Snowflake
Datawarehouse for Asset Management Reporting. SAP Plant Maintenance
data containing Notifications, Orders, master data are loaded into CSVs in
Azure Blob containers. These CSVs then read by Python scripts to load into
import layer, perform calculations/versioning in raw layer and then load into
reporting layer to be used by Tableau reports.


Project 3: Fuel & Energy Consumption Reporting at Plants

  • Design, Develop and Maintain ETL/Data pipelines using Talend & Azure Data Factory ETL tools and Python.
  • Analysis of Source Systems in order to find required data for reports, creating mapping data required for data enrichments, etc.
  • Extract Transform and Load data from Sources Systems such as Azure Data Lake Storage and Azure Blob Containers to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, etc.
  • Creation of data pipelines, data flows, linked services, activities, triggers, etc to move/transform data.
  • Uses various data flow transformations such as lookup, filter, join, exists, conditional split, etc.
  • Creation of Azure Data Factory pipelines to load data into Azure SQL Database, Azure Blob Storage, etc.
  • Creation of Talend ETL data pipelines to read data from SAP, SQL Server, Excel & REST APIs and load into Azure Blob Storage and Snowflake.
  • Creation of Talend data pipelines with components like tRESTClient, tMap, tparallelize, tJava, tDBConnection for snowflake, tAzureStorageConnection for Azure Blobs, etc.
  • Creation of various Snowflake database objects such as schemas, tables etc.
  • Development of data pipelines using python and Snow SQLs to read data staging layer, perform various validations/transformations/calculations, etc.
  • Development of Snow SQLs for enrichment and transformation of data from Import layer to raw layer and further to reporting layer to be used by reporting tools like Tableau.
  • Development of Python scripts to connect to Snowflake and execute Snow SQLs on Snowflake data warehouse.
  • Development of Python jobs using various data engineering packages like Pandas, SQLAlchemy,etc for various data transformations.
  • Deployment of Python jobs to Azure Batch Account-Applications to be executed by Jobs in Azure Batch Account.
  • Creation of Talend jobs to load various output files to Azure Storage Account Containers to be further processed by python to load into Snowflake.
  • Creation of documentation of various processes, data models, data flow diagrams, ETL Architecture on Confluence.
  • Participating in various SCRUM meetings for creating user stories, estimation, backlog grooming, retrospective, etc.
  • Configuration of GIT Repositories for various environments and releases.
  • Deployment of python scripts to nodes in batch account.
  • Creation of Azure Key Vault, credentials and integrating it with Python scripts for retrieving the secrets or credentials.
  • Creation of Pools in Batch account for individual project and deploying applications to batch accounts.
  • Creation of code repositories in Azure DevOps and developing CI/CD release pipelines using classical and YAML way to deploy application automatically to batch accounts.
Microsoft Azure Azure Blobs REST API MS SQL Server Snowflake CSV Excel XML Microsoft Azure SQL DB alend Data Integration Azure Data Factory V2 Python T-SQL SnowSQL Azure Data Factory Triggers Talend Management Console Azure Data Explorer Azure Data Studio
through 1st Solution Consulting Gmbh
2 Jahre 1 Monat
2020-09 - 2022-09

Engergy Data Lake

Data Engineer Python SQL
Data Engineer

Employment Type: Freelance Contract
Role: Data Engineer
Project: Energy Data Lake

Project Technology Stack

Cloud Platform: Microsoft Azure
Source System: Azure Blobs, REST API, MS SQL Server, Snowflake, CSV, Excel, XML, etc.
Target System: Microsoft Azure SQL DB, MS SQL Server, Snowflake, CSV

ETL Tool/Programming Language: Talend Data Integration, Azure Data Factory V2, Python
Other programming languages: Python, T-SQL, SnowSQL

Scheduling Tool: Azure Data Factory Triggers, Talend Management Console

Other Azure tools: Azure Data Explorer, Azure Data Studio

 

Project Details:

 

Project 1:

 

Health, Safety, Security and the Environment (HSSE) Reporting Migration from SQL Server to Snowflake Datawarehouse

 

This project involves extracting incident or cases data from Incident Management Systems via REST APIs. These incident or cases are recorded at various energy assets such as powerplants, units, weir, etc. The extracted data then enriched with central asset master data and then loaded into report tables for generating HSSE Reports in Tableau.

 

Project 2:

 

Asset Management Reporting

 

This project involves extracting data from Azure Blob containers to Snowflake Datawarehouse for Asset Management Reporting. SAP Plant Maintenance data containing Notifications, Orders, master data are loaded into CSVs in Azure Blob containers. These CSVs then read by Python scripts to load into import layer, perform calculations/versioning in raw layer and then load into reporting layer to be used by Tableau reports.

 

Project 3: Fuel & Energy Consumption Reporting at Plants


Azure Talend Databricks
Python SQL
Remote
11 Monate
2020-02 - 2020-12

Regulatory Reporting

ETL Entwickler Informatica PowerCenter
ETL Entwickler

Senior ETL Developer
Credit Suisse
Credit Suisse, Zürich through Atyeti Inc

 

Contract Type: Contract
Role: ETL Developer
Project: Trade & Transaction Regulatory Reporting (TCIS/TAPI)
MIFIR/EMIR Transaction Regulatory Reporting to various LCAs.

Project Technology Stack
Source System: XML Files, Flat Files, Oracle
Target System: Oracle 19c, XML, CSV
ETL Tool: Informatica PowerCenter 10.2
Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting
Scheduling Tool: Control-M

 

Project Details:

 

TAPI (Trading & Product Information) or TCIS (Trading & Sales Controlling & Information Systems is a central EOD repository holding Credit Suisse?s transactions, positions, cash, stock, accruals and valuation data. The data will be fed from all front office systems and back office systems holding Swiss Trads or Positions. Reference data will be taken from other strategic applications such as GRD Product, GRD Customer, MDS, etc. This repository will supply consistent and reconciled data for legal, regulatory and management reporting.

Responsibilities
- Design, Develop and Maintain Informatica ETL/Data pipelines.

- Analysis of JIRA tickets for finding issues and providing ETL/SQL solution

- Development and Support of Enterprise Data Warehouse in loading history data using SCD Type 1 and Type 2 mappings.

- Development of database objects like procedures, functions, views, etc for data loading support.

Oracle 19c
Informatica PowerCenter
Credit Suisse
6 Monate
2020-01 - 2020-06

Procurify Integration with DATEV

Talend Developer
Talend Developer

ETL Developer
Lipsia Digital GmbH

Contract Type: Freelance
Role: ETL Developer
Project: Procurify Integration with DATEV


Read all bill details including purchase orders, approvals, attachments from Procurify, a cloud-based procurement management system and send it to the Flowwer2, a target system for the Procurify DATEV Connector. Flowwer2 is DATEV approved tool which can be connect to a specific DATEV Client and can send via structured data as well as attachments to DATEV.

 

Flowwer2 will be used to receive and send invoice data and related attachments (invoice.pdf, po.pdf, shipping slip.pdf AND approval log.pdf) to DATEV.

Project Technology Stack
Source System: REST API
Target System: PostgreSQL 10.7, REST API
ETL Tool: Talend Open Studio 7.2

Other programming languages: SQL, Unix Shell Scripting
Scheduling Tool: CronTab

Other tools: Github, JIRA, Confluence, PostMan, Putty, WinSCP, etc.

Responsibilities
- Architect, Design, Develop and Maintain Talend jobs.
- Bug Fixing, Deployment, Production Support, Data Analysis
- Read data from REST APIs via components like tRESTClient, tXMLMap, tMap, tPostgresqlInput, etc and load into staging layer in PostgreSQL database using components like tFilterRow, tSortRow, tPostgresqlOutput.
- Perform various cleansing and data completeness checks

- Analysis of JIRA tickets for finding issues and providing ETL/SQL solution
- Used various components like tAggregateRow, tFileFetch, tFileInputDelimited, tFileOutputDelimited, tFileInputRaw, tJava, tJavaRow, tS3Connection, tS3Put, tFlowToIterate, etc.

- Used Talend ESB components like tRESTClient to interact with RESTful Web Service providers by sending HTTP & HTTPS request and processing response received in JSON or XML.

- Create PDF files out of deliver area using tFileOutputPDF2 component from Talend Exchange.

- Created incremental loading components using tSetGlobalVar. Used tContextLoad component to load context variable for loading generic configuration.

- Used tFileFetch component to download PDF documents from URLs.

Talend PostgreSQL REST
Lipsia Digital GmbH
Leipzig
5 Monate
2019-08 - 2019-12

Regulatory Reporting Hub

ETL Entwickler Informatica PowerCenter
ETL Entwickler

Deutsche Boerse, Frankfurt am Main through Marlin Green Ltd

Vertragsart: Freiberuflich

Role: ETL Entwickler

Project: Regulatory Reporting Hub (RRH)

MIFIR/EMIR Transaction Regulatory Reporting to NCAs e.g. BaFin, AMF, etc.

Project Technology Stack

Source System: XML Files, Flat Files, Oracle

Target System: Oracle, XML, CSV

ETL Tool: Informatica Powercenter 10.2

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Control-M

 

Responsibilities

- Design, Develop and Maintain Informatica ETL/Data pipelines

- Performance tuning of ETL pipelines for faster loading in various environments

- Bug Fixing, Deployment, Production Support, Data Analysis

- Read data from XML & Flat files to load into staging, core layer and further to Delivery Area in Oracle database.

- Perform various cleansing and data completeness checks

- Enrich data from various reference/lookup tables and load into core layer

- Used various transformation like XML Source Qualifier, XML Parser, XML Generator, Transaction Control, Normalizer, lookup, update strategy, etc.

- Performance optimization of informatica mappings and sessions for faster loads

- Developed SCD Type1 and 2 mappings to load history data into data mart.

Informatica PowerCenter
Informatica PowerCenter
Deutsche Boerse
Frankfurt am Main
1 Jahr 9 Monate
2017-10 - 2019-06

Compliance - AML Reporting

ETL Entwickler
ETL Entwickler

Commerzbank, Frankfurt am Main through JOB AG Source One GmbH

Vertragsart: Freiberuflich

Role: ETL Entwickler

Project

Compliance (CMC & CAF) - AML Reporting - Frankfurt & Singapore

This was a data Integration project which includes providing data from various banking applications like Murex Cash, Murex Equity, Murex Currency, etc. for compliance reporting.

Project Technology Stack

Source System: Flat Files, MS SQL Server

Target System: Oracle, Flat Files, Hadoop HDFS

ETL Tool: Informatica Powercenter 10.1, Informatica BDM

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting, UC4 Scripting

Scheduling Tool: Automic UC4

 

Responsibilities:

 

-         Design ETL Pipelines and ETL Architecture.

-         Design Informatica ETL jobs as per the quality and software development standards.

-         Source to target data mapping analysis and design.

-         Analyze, design, develop, test and document Informatica ETL programs from detailed and high-level specifications, and assist in troubleshooting.

-         Creation of project-related documents like HLD, LLD, etc.

-         Created reusable transformations and mapplets

-         Developed data ETL pipelines for Change Data Capture (CDC)

-         Creation of data pipelines to load into Hadoop HDFS.

-         Complex Informatica Powercenter ETL development and Quality Assurance.

-         Design and develop various slowly changing dimension load e.g. Type 1, Type 2, and Type 3

-         Responsible for finding various bottlenecks and performance tuning at various levels like mapping level, session level, and database level.

-         Extensive use of various active and passive transformations like Filter, Router, Expression, Source Qualifier, Joiner, and Look up, Update Strategy, Sequence Generator, Rank, and Aggregator.

-         Debugging and troubleshooting Sessions using the Informatica Debugger and Workflow Monitor.

-         Implement various loads like Daily Loads, Weekly Loads, and Quarterly Loads.

-         Conduct Unit tests, Integration tests, performance tests, etc.

-         Contact point for problems in the Production environment and Defects Tracking with business. (3rd-Level-Support)

-         Supported deployment team in various environment's deployments

-         Developed database objects including tables, Indexes, views, sequences, packages, triggers and procedures to troubleshoot any database problems

Informatica PowerCenter Microsoft SQL Server 2008 R2 Oracle Oracle SQL UC4 Software
Frankfurt
2 Jahre 4 Monate
2015-06 - 2017-09

Retail Enterprise Data Warehouse

ETL Tech Lead
ETL Tech Lead

Aldi Sued, Muelhiem an der Ruhr through Templeton & Partners Ltd

Vertragsart: Freiberuflich

Role: ETL Tech Lead

Project: Retail Enterprise Data Warehouse

Project Technology Stack

Source System: MS SQL Server, Flat Files, Oracle

Target System: Oracle Exadata

ETL Tool: Informatica Powercenter 10.1

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Informatica Scheduler

Project Methodology: Scrum/Agile

 

Responsibilities

-         Participate in scoping, data quality analysis, source system data analysis, target system requirements, volume analysis and migration window determination.

-         Implement various loads like Daily Loads, Weekly Loads, and Quarterly Loads.

-         Perform data cleansing tasks.

-         Perform test using sample test data in accordance with the client data migration/integration needs.

-         Contact point for problems in the Production environment and Defects Tracking with business. (3rd-Level-Support)

-         Helped Business Analyst in refining mapping specification documents.

-         Developed Informatica Powercenter mappings to move data from stage to target tables

-         Developed PL/SQL Packages, Procedures and Functions accordance with Business Requirements.

-         Documented various input databases and data sources.

-         Debugging and troubleshooting Sessions using the Informatica Debugger and Workflow Monitor.

-         Complex ETL development and Quality Assurance.

-         Responsible for finding various bottlenecks and performance tuning at various levels like database, ETL, etc.

-         Created Materialized Views and partitioning tables for performance reasons.

-         Worked on various back end Procedures and Functions using PL/SQL.

-         Developed UNIX shell scripts to perform various user requirements.

-         Designing Tables, Constraints, Views, and Indexes etc.

-         Developed database objects including tables, Indexes, views, sequences, packages, triggers and procedures to troubleshoot any database problems

-         Tuned complex Stored Procedures for faster execution

-         Responsible for Analyzing and Implementing the Change Requests.

-         Involved in handling the changes in compiling jobs and scripts according to the database changes.

Informatica PowerCenter
Aldi Sued
Muelhiem an der Ruhr
5 Monate
2015-01 - 2015-05

Hotel Enterprise Data Warehouse

Senior ETL Consultant
Senior ETL Consultant

HRS, Köln through Informationsfabrik GmbH

Job Type:
Freelancer

Role: Senior ETL Consultant

Project: Hotel Enterprise Data Warehouse

Project Technology Stack

Source System: MS SQL Server, Flat Files, Oracle, XML

Target System: Sybase IQ

ETL Tool: Informatica Powercenter 9.5

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Control-M

Project Methodology: Waterfall

Data Modeling: Data Vault

HRS
Köln
8 Monate
2014-05 - 2014-12

Karstadt information systems for measures and analytics (KARISMA)

ETL Tech Lead
ETL Tech Lead

Karstadt, Essen through IBM Deutschland GmbH through Questax Heidelberg GmbH

Job Type:
Freelancer

Role: ETL Tech Lead

Project:

Karstadt information systems for measures and analytics (KARISMA)

The goal of this project was to create centralized Analytical and Reporting system for Karstadt Warehouse GmbH. The major part of the project was to replace existing SAP BW Reporting system and create new enterprise data warehouse with Informatica PowerCenter 9.5.1 for ETL and Cognos 10 for Reporting. Informatica PowerExchange 9.5.1 with BCI (Business Content Integration) & Data Integration using ABAP methods were used to connect to Karstadt SAP Retail system and read data from SAP Standard and Customized Data Sources. IBM Netezza 7 was used as Target system with Informatica PowerExchange for Netezza.

Project Technology Stack

Source System: SAP, IDOC, Flat Files, XML

Target System: IBM Netezza

ETL Tool: Informatica Powercenter 9.5, Informatica Powerexchange 9.5

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Informatica Scheduler

Project Methodology: Waterfall

Karstadt
Essen, Ruhr
4 Monate
2014-01 - 2014-04

Data Integration

Senior ETL Consultant
Senior ETL Consultant

Deutsche Boerse, Frankfurt through Javaji Softech GmbH & Co. KG

Job Type: Freelancer

Role: Senior ETL Consultant

Project: Data Integration

Deutsche Boerse
Frankfurt am Main
2 Jahre 6 Monate
2011-07 - 2013-12

Postbank Savings Deposit Accounts Migration

Senior ETL Consultant
Senior ETL Consultant

Deutsche Bank, Frankfurt through Datamatics Global Solutions GmbH/DXC GmbH (Formerly CSC Deutschland) and Hays AG

Job Type: Employee & Freelancer

Role: Senior ETL Consultant

Informatica Powercenter ETL Tool Development & Support
for Data Migration and Data Integration Projects.

Projects:
#1 Retail Banking - Postbank Savings Deposit Accounts Migration
#2 Retail Banking - Postbank Savings Deposit Accounts Integration

#3 Retail Banking - Auto Deployment

#4 Retail Banking - LDAP Integration

Deutsche Bank
Frankfurt am Main
9 Monate
2010-11 - 2011-07

Home Mortgage Enterprise Data Warehouse

Senior ETL Consultant
Senior ETL Consultant

American Home Mortgage Servicing Inc, Texas through Hitachi Consulting Pvt Ltd, Pune

Job Type: Employee

Role: Senior ETL Consultant

Project :

Enterprise Data Warehouse

Informatica PowerCenter
American Home Mortgage Servicing Inc
Texas
2 Jahre 1 Monat
2008-10 - 2010-10

SMP

Software Engineer
Software Engineer

Sigma Systems, Pune

Job Type: Employee

Role: Software Engineer

Oracle, Unix, Java Development & Support

Sigma Systems
Pune

Aus- und Weiterbildung

Aus- und Weiterbildung

Bachelor Of Engineering in Computer Science

Pune University, Pune, India


ZERTIFIZIERUNGEN

March 2014

Informatica PowerCenter 9.x Certified Professional

Certificate No. 004-000384

Percentile: 71%

May 2015

TOGAF 9.1 (Enterprise Architecture Framework) Certified Professional from OPEN GROUP

Certification ID. 96457

Percentile: 75%

July 2017

International Knowledge Mesurement

Certificate: Informatica Powercenter

Percentile: 93%

Kompetenzen

Kompetenzen

Top-Skills

Python Data Engineering Azure Oracle/SQL Data Integration Python AWS azure data factory Databricks Snowflake ETL Azure Data Engineer Python Pandas talend apache spark

Produkte / Standards / Erfahrungen / Methoden

Apache Hadoop
Automic Software
ETL
Informatica PowerCenter
Informatica PowerExchange
Java
Oracle
Pentaho
REST
Talend
UC4 Software
XML

ETL Tools

Informatica PowerCenter, Informatica Big Data Management,

Informatica Power Exchange, Talend

 

Databases

Oracle 12c, Oracle Exadata 12c, Microsoft SQL Server 2016, Hadoop HDFS, XML

 

Big Data Technologies/Ecosystem

Cloudera, HDFS, YARN, MapReduce, Hive, Pig, HBase, Oozie, Flume and Sqoop

Modeling

Star & Snowflake Schema, 3-NF, Data Modeling, Dimensional Modeling, Data Vault

 

Modeling Tools

PowerDesinger, Informatica Mapping Architect for Visio

 

Software Development Methods

Agile, SCRUM, Waterfall

 

Programming Languages

Core JAVA, SQL, T-SQL, PL/SQL, UNIX/Bash Shell scripting

Scheduler

BMC Control-M, Automic UC4, Informatica Scheduler

Version Control

Informatica Version Control, Subversion, Tortoise SVN, GitHub

Other Tools - Atlassian Jira, Atlassian Confluence, GitHub, Hue, Eclipse, Toad, PL/SQL-Developer, FTP, sFTP, WinSCP, FileZilla, Putty, HP Quality Center, Aginity Workbench for IBM Netezza

 

 

  • Having 10+ years of IT Consulting experience with emphasis on business requirements analysis, ETL development, implementations, testing and production support of Data Lake, Data Migration, Data Warehousing and Business Intelligence projects.
  • Expert in development and designing of ETL methodology for supporting data transformations using Informatica Power Center 9.5.1, Power Exchange 9.5.1
  • Experience in creating Proof of Concepts using Informatica PowerCenter and PowerExchange.
  • Have experience in agile ETL Development using SCRUM method.
  • Have experience in ETL design, ETL architecture using Informatica PowerCenter.
  • Have experience in Informatica Mapping Architect for Visio for creating mapping templates for each Data Layer in DW.
  • Have experience in Dimensional modeling and data modeling.
  • Highly Skilled in ETL & Database Performance Optimizations.
  • Experienced in analyzing business requirements and translating functional requirements into technical design specifications like LLDs, JHBs.
  • Experience in reading and writing data into Mainframe systems using Informatica Power Exchange 9.1
  • Experience in Informatica Powercenter administration tasks like installation, code migration, etc.
  • Worked with different data sources like Oracle, SQL Server, SAP BW, Mainframe Data sets, Flat Files, XML files, JMS Queues and MQ queues.
  • Hands on experience in writing, testing and implementation of the views, triggers, stored procedures, functions, packages at database level using SQL, PL/SQL.
  • Experience in relational database development using Oracle PL/SQL and SQL.
  • Extensively worked on UNIX environments and Shell Script development.
  • Hands on experience in handling large volumes of data in production environments.
  • Worked on Unit Testing, Integration testing and Regression testing to verify load order, time window and lookup with full load.
  • Experience in leading onsite development team.
  • Excellent aptitude to understand and meet client needs.
  • Excellent communication, presentation, project management, leadership skills and a very good team player and self-starter with ability to work independently and as part of a team.

Betriebssysteme

UNIX (Sun Solaris, Linux), Windows 7/Vista/XP

Programmiersprachen

Bash Shell
Java
Oracle PLSQL
Oracle SQL

Datenbanken

HDFS
IBM DB2
Microsoft SQL Server 2008 R2
Microsoft SQL Server 2008 R2
Oracle
Oracle 19c
PostgreSQL
Sybase (ASE)

Design / Entwicklung / Konstruktion

ETL Design, ETL Development using Informatica PowerCenter

Branchen

Branchen

Banking & Finance, Telecom

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

3 Monate
2023-02 - 2023-04

more projects

Microsoft Azure Azure Blobs REST API ...
Project 1:
Health, Safety, Security and the Environment (HSSE) Reporting Migration from
SQL Server to Snowflake Datawarehouse
This project involves extracting incident or cases data from Incident
Management Systems via REST APIs. These incident or cases are recorded at
various energy assets such as powerplants, units, weir, etc. The extracted data
then enriched with central asset master data and then loaded into report tables
for generating HSSE Reports in Tableau.


Project 2:Asset Management Reporting

This project involves extracting data from Azure Blob containers to Snowflake
Datawarehouse for Asset Management Reporting. SAP Plant Maintenance
data containing Notifications, Orders, master data are loaded into CSVs in
Azure Blob containers. These CSVs then read by Python scripts to load into
import layer, perform calculations/versioning in raw layer and then load into
reporting layer to be used by Tableau reports.


Project 3: Fuel & Energy Consumption Reporting at Plants

  • Design, Develop and Maintain ETL/Data pipelines using Talend & Azure Data Factory ETL tools and Python.
  • Analysis of Source Systems in order to find required data for reports, creating mapping data required for data enrichments, etc.
  • Extract Transform and Load data from Sources Systems such as Azure Data Lake Storage and Azure Blob Containers to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, etc.
  • Creation of data pipelines, data flows, linked services, activities, triggers, etc to move/transform data.
  • Uses various data flow transformations such as lookup, filter, join, exists, conditional split, etc.
  • Creation of Azure Data Factory pipelines to load data into Azure SQL Database, Azure Blob Storage, etc.
  • Creation of Talend ETL data pipelines to read data from SAP, SQL Server, Excel & REST APIs and load into Azure Blob Storage and Snowflake.
  • Creation of Talend data pipelines with components like tRESTClient, tMap, tparallelize, tJava, tDBConnection for snowflake, tAzureStorageConnection for Azure Blobs, etc.
  • Creation of various Snowflake database objects such as schemas, tables etc.
  • Development of data pipelines using python and Snow SQLs to read data staging layer, perform various validations/transformations/calculations, etc.
  • Development of Snow SQLs for enrichment and transformation of data from Import layer to raw layer and further to reporting layer to be used by reporting tools like Tableau.
  • Development of Python scripts to connect to Snowflake and execute Snow SQLs on Snowflake data warehouse.
  • Development of Python jobs using various data engineering packages like Pandas, SQLAlchemy,etc for various data transformations.
  • Deployment of Python jobs to Azure Batch Account-Applications to be executed by Jobs in Azure Batch Account.
  • Creation of Talend jobs to load various output files to Azure Storage Account Containers to be further processed by python to load into Snowflake.
  • Creation of documentation of various processes, data models, data flow diagrams, ETL Architecture on Confluence.
  • Participating in various SCRUM meetings for creating user stories, estimation, backlog grooming, retrospective, etc.
  • Configuration of GIT Repositories for various environments and releases.
  • Deployment of python scripts to nodes in batch account.
  • Creation of Azure Key Vault, credentials and integrating it with Python scripts for retrieving the secrets or credentials.
  • Creation of Pools in Batch account for individual project and deploying applications to batch accounts.
  • Creation of code repositories in Azure DevOps and developing CI/CD release pipelines using classical and YAML way to deploy application automatically to batch accounts.
Microsoft Azure Azure Blobs REST API MS SQL Server Snowflake CSV Excel XML Microsoft Azure SQL DB alend Data Integration Azure Data Factory V2 Python T-SQL SnowSQL Azure Data Factory Triggers Talend Management Console Azure Data Explorer Azure Data Studio
through 1st Solution Consulting Gmbh
2 Jahre 1 Monat
2020-09 - 2022-09

Engergy Data Lake

Data Engineer Python SQL
Data Engineer

Employment Type: Freelance Contract
Role: Data Engineer
Project: Energy Data Lake

Project Technology Stack

Cloud Platform: Microsoft Azure
Source System: Azure Blobs, REST API, MS SQL Server, Snowflake, CSV, Excel, XML, etc.
Target System: Microsoft Azure SQL DB, MS SQL Server, Snowflake, CSV

ETL Tool/Programming Language: Talend Data Integration, Azure Data Factory V2, Python
Other programming languages: Python, T-SQL, SnowSQL

Scheduling Tool: Azure Data Factory Triggers, Talend Management Console

Other Azure tools: Azure Data Explorer, Azure Data Studio

 

Project Details:

 

Project 1:

 

Health, Safety, Security and the Environment (HSSE) Reporting Migration from SQL Server to Snowflake Datawarehouse

 

This project involves extracting incident or cases data from Incident Management Systems via REST APIs. These incident or cases are recorded at various energy assets such as powerplants, units, weir, etc. The extracted data then enriched with central asset master data and then loaded into report tables for generating HSSE Reports in Tableau.

 

Project 2:

 

Asset Management Reporting

 

This project involves extracting data from Azure Blob containers to Snowflake Datawarehouse for Asset Management Reporting. SAP Plant Maintenance data containing Notifications, Orders, master data are loaded into CSVs in Azure Blob containers. These CSVs then read by Python scripts to load into import layer, perform calculations/versioning in raw layer and then load into reporting layer to be used by Tableau reports.

 

Project 3: Fuel & Energy Consumption Reporting at Plants


Azure Talend Databricks
Python SQL
Remote
11 Monate
2020-02 - 2020-12

Regulatory Reporting

ETL Entwickler Informatica PowerCenter
ETL Entwickler

Senior ETL Developer
Credit Suisse
Credit Suisse, Zürich through Atyeti Inc

 

Contract Type: Contract
Role: ETL Developer
Project: Trade & Transaction Regulatory Reporting (TCIS/TAPI)
MIFIR/EMIR Transaction Regulatory Reporting to various LCAs.

Project Technology Stack
Source System: XML Files, Flat Files, Oracle
Target System: Oracle 19c, XML, CSV
ETL Tool: Informatica PowerCenter 10.2
Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting
Scheduling Tool: Control-M

 

Project Details:

 

TAPI (Trading & Product Information) or TCIS (Trading & Sales Controlling & Information Systems is a central EOD repository holding Credit Suisse?s transactions, positions, cash, stock, accruals and valuation data. The data will be fed from all front office systems and back office systems holding Swiss Trads or Positions. Reference data will be taken from other strategic applications such as GRD Product, GRD Customer, MDS, etc. This repository will supply consistent and reconciled data for legal, regulatory and management reporting.

Responsibilities
- Design, Develop and Maintain Informatica ETL/Data pipelines.

- Analysis of JIRA tickets for finding issues and providing ETL/SQL solution

- Development and Support of Enterprise Data Warehouse in loading history data using SCD Type 1 and Type 2 mappings.

- Development of database objects like procedures, functions, views, etc for data loading support.

Oracle 19c
Informatica PowerCenter
Credit Suisse
6 Monate
2020-01 - 2020-06

Procurify Integration with DATEV

Talend Developer
Talend Developer

ETL Developer
Lipsia Digital GmbH

Contract Type: Freelance
Role: ETL Developer
Project: Procurify Integration with DATEV


Read all bill details including purchase orders, approvals, attachments from Procurify, a cloud-based procurement management system and send it to the Flowwer2, a target system for the Procurify DATEV Connector. Flowwer2 is DATEV approved tool which can be connect to a specific DATEV Client and can send via structured data as well as attachments to DATEV.

 

Flowwer2 will be used to receive and send invoice data and related attachments (invoice.pdf, po.pdf, shipping slip.pdf AND approval log.pdf) to DATEV.

Project Technology Stack
Source System: REST API
Target System: PostgreSQL 10.7, REST API
ETL Tool: Talend Open Studio 7.2

Other programming languages: SQL, Unix Shell Scripting
Scheduling Tool: CronTab

Other tools: Github, JIRA, Confluence, PostMan, Putty, WinSCP, etc.

Responsibilities
- Architect, Design, Develop and Maintain Talend jobs.
- Bug Fixing, Deployment, Production Support, Data Analysis
- Read data from REST APIs via components like tRESTClient, tXMLMap, tMap, tPostgresqlInput, etc and load into staging layer in PostgreSQL database using components like tFilterRow, tSortRow, tPostgresqlOutput.
- Perform various cleansing and data completeness checks

- Analysis of JIRA tickets for finding issues and providing ETL/SQL solution
- Used various components like tAggregateRow, tFileFetch, tFileInputDelimited, tFileOutputDelimited, tFileInputRaw, tJava, tJavaRow, tS3Connection, tS3Put, tFlowToIterate, etc.

- Used Talend ESB components like tRESTClient to interact with RESTful Web Service providers by sending HTTP & HTTPS request and processing response received in JSON or XML.

- Create PDF files out of deliver area using tFileOutputPDF2 component from Talend Exchange.

- Created incremental loading components using tSetGlobalVar. Used tContextLoad component to load context variable for loading generic configuration.

- Used tFileFetch component to download PDF documents from URLs.

Talend PostgreSQL REST
Lipsia Digital GmbH
Leipzig
5 Monate
2019-08 - 2019-12

Regulatory Reporting Hub

ETL Entwickler Informatica PowerCenter
ETL Entwickler

Deutsche Boerse, Frankfurt am Main through Marlin Green Ltd

Vertragsart: Freiberuflich

Role: ETL Entwickler

Project: Regulatory Reporting Hub (RRH)

MIFIR/EMIR Transaction Regulatory Reporting to NCAs e.g. BaFin, AMF, etc.

Project Technology Stack

Source System: XML Files, Flat Files, Oracle

Target System: Oracle, XML, CSV

ETL Tool: Informatica Powercenter 10.2

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Control-M

 

Responsibilities

- Design, Develop and Maintain Informatica ETL/Data pipelines

- Performance tuning of ETL pipelines for faster loading in various environments

- Bug Fixing, Deployment, Production Support, Data Analysis

- Read data from XML & Flat files to load into staging, core layer and further to Delivery Area in Oracle database.

- Perform various cleansing and data completeness checks

- Enrich data from various reference/lookup tables and load into core layer

- Used various transformation like XML Source Qualifier, XML Parser, XML Generator, Transaction Control, Normalizer, lookup, update strategy, etc.

- Performance optimization of informatica mappings and sessions for faster loads

- Developed SCD Type1 and 2 mappings to load history data into data mart.

Informatica PowerCenter
Informatica PowerCenter
Deutsche Boerse
Frankfurt am Main
1 Jahr 9 Monate
2017-10 - 2019-06

Compliance - AML Reporting

ETL Entwickler
ETL Entwickler

Commerzbank, Frankfurt am Main through JOB AG Source One GmbH

Vertragsart: Freiberuflich

Role: ETL Entwickler

Project

Compliance (CMC & CAF) - AML Reporting - Frankfurt & Singapore

This was a data Integration project which includes providing data from various banking applications like Murex Cash, Murex Equity, Murex Currency, etc. for compliance reporting.

Project Technology Stack

Source System: Flat Files, MS SQL Server

Target System: Oracle, Flat Files, Hadoop HDFS

ETL Tool: Informatica Powercenter 10.1, Informatica BDM

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting, UC4 Scripting

Scheduling Tool: Automic UC4

 

Responsibilities:

 

-         Design ETL Pipelines and ETL Architecture.

-         Design Informatica ETL jobs as per the quality and software development standards.

-         Source to target data mapping analysis and design.

-         Analyze, design, develop, test and document Informatica ETL programs from detailed and high-level specifications, and assist in troubleshooting.

-         Creation of project-related documents like HLD, LLD, etc.

-         Created reusable transformations and mapplets

-         Developed data ETL pipelines for Change Data Capture (CDC)

-         Creation of data pipelines to load into Hadoop HDFS.

-         Complex Informatica Powercenter ETL development and Quality Assurance.

-         Design and develop various slowly changing dimension load e.g. Type 1, Type 2, and Type 3

-         Responsible for finding various bottlenecks and performance tuning at various levels like mapping level, session level, and database level.

-         Extensive use of various active and passive transformations like Filter, Router, Expression, Source Qualifier, Joiner, and Look up, Update Strategy, Sequence Generator, Rank, and Aggregator.

-         Debugging and troubleshooting Sessions using the Informatica Debugger and Workflow Monitor.

-         Implement various loads like Daily Loads, Weekly Loads, and Quarterly Loads.

-         Conduct Unit tests, Integration tests, performance tests, etc.

-         Contact point for problems in the Production environment and Defects Tracking with business. (3rd-Level-Support)

-         Supported deployment team in various environment's deployments

-         Developed database objects including tables, Indexes, views, sequences, packages, triggers and procedures to troubleshoot any database problems

Informatica PowerCenter Microsoft SQL Server 2008 R2 Oracle Oracle SQL UC4 Software
Frankfurt
2 Jahre 4 Monate
2015-06 - 2017-09

Retail Enterprise Data Warehouse

ETL Tech Lead
ETL Tech Lead

Aldi Sued, Muelhiem an der Ruhr through Templeton & Partners Ltd

Vertragsart: Freiberuflich

Role: ETL Tech Lead

Project: Retail Enterprise Data Warehouse

Project Technology Stack

Source System: MS SQL Server, Flat Files, Oracle

Target System: Oracle Exadata

ETL Tool: Informatica Powercenter 10.1

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Informatica Scheduler

Project Methodology: Scrum/Agile

 

Responsibilities

-         Participate in scoping, data quality analysis, source system data analysis, target system requirements, volume analysis and migration window determination.

-         Implement various loads like Daily Loads, Weekly Loads, and Quarterly Loads.

-         Perform data cleansing tasks.

-         Perform test using sample test data in accordance with the client data migration/integration needs.

-         Contact point for problems in the Production environment and Defects Tracking with business. (3rd-Level-Support)

-         Helped Business Analyst in refining mapping specification documents.

-         Developed Informatica Powercenter mappings to move data from stage to target tables

-         Developed PL/SQL Packages, Procedures and Functions accordance with Business Requirements.

-         Documented various input databases and data sources.

-         Debugging and troubleshooting Sessions using the Informatica Debugger and Workflow Monitor.

-         Complex ETL development and Quality Assurance.

-         Responsible for finding various bottlenecks and performance tuning at various levels like database, ETL, etc.

-         Created Materialized Views and partitioning tables for performance reasons.

-         Worked on various back end Procedures and Functions using PL/SQL.

-         Developed UNIX shell scripts to perform various user requirements.

-         Designing Tables, Constraints, Views, and Indexes etc.

-         Developed database objects including tables, Indexes, views, sequences, packages, triggers and procedures to troubleshoot any database problems

-         Tuned complex Stored Procedures for faster execution

-         Responsible for Analyzing and Implementing the Change Requests.

-         Involved in handling the changes in compiling jobs and scripts according to the database changes.

Informatica PowerCenter
Aldi Sued
Muelhiem an der Ruhr
5 Monate
2015-01 - 2015-05

Hotel Enterprise Data Warehouse

Senior ETL Consultant
Senior ETL Consultant

HRS, Köln through Informationsfabrik GmbH

Job Type:
Freelancer

Role: Senior ETL Consultant

Project: Hotel Enterprise Data Warehouse

Project Technology Stack

Source System: MS SQL Server, Flat Files, Oracle, XML

Target System: Sybase IQ

ETL Tool: Informatica Powercenter 9.5

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Control-M

Project Methodology: Waterfall

Data Modeling: Data Vault

HRS
Köln
8 Monate
2014-05 - 2014-12

Karstadt information systems for measures and analytics (KARISMA)

ETL Tech Lead
ETL Tech Lead

Karstadt, Essen through IBM Deutschland GmbH through Questax Heidelberg GmbH

Job Type:
Freelancer

Role: ETL Tech Lead

Project:

Karstadt information systems for measures and analytics (KARISMA)

The goal of this project was to create centralized Analytical and Reporting system for Karstadt Warehouse GmbH. The major part of the project was to replace existing SAP BW Reporting system and create new enterprise data warehouse with Informatica PowerCenter 9.5.1 for ETL and Cognos 10 for Reporting. Informatica PowerExchange 9.5.1 with BCI (Business Content Integration) & Data Integration using ABAP methods were used to connect to Karstadt SAP Retail system and read data from SAP Standard and Customized Data Sources. IBM Netezza 7 was used as Target system with Informatica PowerExchange for Netezza.

Project Technology Stack

Source System: SAP, IDOC, Flat Files, XML

Target System: IBM Netezza

ETL Tool: Informatica Powercenter 9.5, Informatica Powerexchange 9.5

Other programming languages: Oracle SQL & PLSQL, Unix Shell Scripting

Scheduling Tool: Informatica Scheduler

Project Methodology: Waterfall

Karstadt
Essen, Ruhr
4 Monate
2014-01 - 2014-04

Data Integration

Senior ETL Consultant
Senior ETL Consultant

Deutsche Boerse, Frankfurt through Javaji Softech GmbH & Co. KG

Job Type: Freelancer

Role: Senior ETL Consultant

Project: Data Integration

Deutsche Boerse
Frankfurt am Main
2 Jahre 6 Monate
2011-07 - 2013-12

Postbank Savings Deposit Accounts Migration

Senior ETL Consultant
Senior ETL Consultant

Deutsche Bank, Frankfurt through Datamatics Global Solutions GmbH/DXC GmbH (Formerly CSC Deutschland) and Hays AG

Job Type: Employee & Freelancer

Role: Senior ETL Consultant

Informatica Powercenter ETL Tool Development & Support
for Data Migration and Data Integration Projects.

Projects:
#1 Retail Banking - Postbank Savings Deposit Accounts Migration
#2 Retail Banking - Postbank Savings Deposit Accounts Integration

#3 Retail Banking - Auto Deployment

#4 Retail Banking - LDAP Integration

Deutsche Bank
Frankfurt am Main
9 Monate
2010-11 - 2011-07

Home Mortgage Enterprise Data Warehouse

Senior ETL Consultant
Senior ETL Consultant

American Home Mortgage Servicing Inc, Texas through Hitachi Consulting Pvt Ltd, Pune

Job Type: Employee

Role: Senior ETL Consultant

Project :

Enterprise Data Warehouse

Informatica PowerCenter
American Home Mortgage Servicing Inc
Texas
2 Jahre 1 Monat
2008-10 - 2010-10

SMP

Software Engineer
Software Engineer

Sigma Systems, Pune

Job Type: Employee

Role: Software Engineer

Oracle, Unix, Java Development & Support

Sigma Systems
Pune

Aus- und Weiterbildung

Aus- und Weiterbildung

Bachelor Of Engineering in Computer Science

Pune University, Pune, India


ZERTIFIZIERUNGEN

March 2014

Informatica PowerCenter 9.x Certified Professional

Certificate No. 004-000384

Percentile: 71%

May 2015

TOGAF 9.1 (Enterprise Architecture Framework) Certified Professional from OPEN GROUP

Certification ID. 96457

Percentile: 75%

July 2017

International Knowledge Mesurement

Certificate: Informatica Powercenter

Percentile: 93%

Kompetenzen

Kompetenzen

Top-Skills

Python Data Engineering Azure Oracle/SQL Data Integration Python AWS azure data factory Databricks Snowflake ETL Azure Data Engineer Python Pandas talend apache spark

Produkte / Standards / Erfahrungen / Methoden

Apache Hadoop
Automic Software
ETL
Informatica PowerCenter
Informatica PowerExchange
Java
Oracle
Pentaho
REST
Talend
UC4 Software
XML

ETL Tools

Informatica PowerCenter, Informatica Big Data Management,

Informatica Power Exchange, Talend

 

Databases

Oracle 12c, Oracle Exadata 12c, Microsoft SQL Server 2016, Hadoop HDFS, XML

 

Big Data Technologies/Ecosystem

Cloudera, HDFS, YARN, MapReduce, Hive, Pig, HBase, Oozie, Flume and Sqoop

Modeling

Star & Snowflake Schema, 3-NF, Data Modeling, Dimensional Modeling, Data Vault

 

Modeling Tools

PowerDesinger, Informatica Mapping Architect for Visio

 

Software Development Methods

Agile, SCRUM, Waterfall

 

Programming Languages

Core JAVA, SQL, T-SQL, PL/SQL, UNIX/Bash Shell scripting

Scheduler

BMC Control-M, Automic UC4, Informatica Scheduler

Version Control

Informatica Version Control, Subversion, Tortoise SVN, GitHub

Other Tools - Atlassian Jira, Atlassian Confluence, GitHub, Hue, Eclipse, Toad, PL/SQL-Developer, FTP, sFTP, WinSCP, FileZilla, Putty, HP Quality Center, Aginity Workbench for IBM Netezza

 

 

  • Having 10+ years of IT Consulting experience with emphasis on business requirements analysis, ETL development, implementations, testing and production support of Data Lake, Data Migration, Data Warehousing and Business Intelligence projects.
  • Expert in development and designing of ETL methodology for supporting data transformations using Informatica Power Center 9.5.1, Power Exchange 9.5.1
  • Experience in creating Proof of Concepts using Informatica PowerCenter and PowerExchange.
  • Have experience in agile ETL Development using SCRUM method.
  • Have experience in ETL design, ETL architecture using Informatica PowerCenter.
  • Have experience in Informatica Mapping Architect for Visio for creating mapping templates for each Data Layer in DW.
  • Have experience in Dimensional modeling and data modeling.
  • Highly Skilled in ETL & Database Performance Optimizations.
  • Experienced in analyzing business requirements and translating functional requirements into technical design specifications like LLDs, JHBs.
  • Experience in reading and writing data into Mainframe systems using Informatica Power Exchange 9.1
  • Experience in Informatica Powercenter administration tasks like installation, code migration, etc.
  • Worked with different data sources like Oracle, SQL Server, SAP BW, Mainframe Data sets, Flat Files, XML files, JMS Queues and MQ queues.
  • Hands on experience in writing, testing and implementation of the views, triggers, stored procedures, functions, packages at database level using SQL, PL/SQL.
  • Experience in relational database development using Oracle PL/SQL and SQL.
  • Extensively worked on UNIX environments and Shell Script development.
  • Hands on experience in handling large volumes of data in production environments.
  • Worked on Unit Testing, Integration testing and Regression testing to verify load order, time window and lookup with full load.
  • Experience in leading onsite development team.
  • Excellent aptitude to understand and meet client needs.
  • Excellent communication, presentation, project management, leadership skills and a very good team player and self-starter with ability to work independently and as part of a team.

Betriebssysteme

UNIX (Sun Solaris, Linux), Windows 7/Vista/XP

Programmiersprachen

Bash Shell
Java
Oracle PLSQL
Oracle SQL

Datenbanken

HDFS
IBM DB2
Microsoft SQL Server 2008 R2
Microsoft SQL Server 2008 R2
Oracle
Oracle 19c
PostgreSQL
Sybase (ASE)

Design / Entwicklung / Konstruktion

ETL Design, ETL Development using Informatica PowerCenter

Branchen

Branchen

Banking & Finance, Telecom

Vertrauen Sie auf GULP

Im Bereich Freelancing
Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

Fragen?

Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Das GULP Freelancer-Portal

Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.