• Led a team of architect & developers to create Delta Lakehouse on Databricks.
• Helped the organization to lower the BI cost by implementing solutions using Power BI and Databricks.
• Evaluated capabilities and limitations of the Databricks and Power BI and created architecture which can scale and respond to user queries in seconds over large dataset. (30 seconds of SLA over 40Gb of fact table).
• Took decisions to create solution which can satisfy all the technical, business and process constraints.
• Designed and created multiple data pipeline for data extraction, cleansing andmapping business logic using spark API for Databricks.
• Fully understand the Databricks data science, ML and BI offerings, compute resources, SQL Warehouse, analytical engine (spark, catalyst, photon) etc. and able to create multi-cloud, hybrid BI solutions.

• Designed and developed data pipeline using Google cloud services (Docker, python, GCS, GKE, Cloud Compose, Big Table, Big Query, Cloud function, and scheduler)
• Created real-time vehicle tracking application for route planning, passenger, and fleet management for Nuremberg Public transport association (VGN).
• Independently created analytics platform, which can scale based on the demand. based on Apache NiFi, Mongo DB, and MySQL.
• The platform was capable to receive real time streaming data from 10000 devices from 2000 locations.
• Used mongo dB to store over 4 Tb of large JSON Objects and MySQL to store Facts, Dimensions, and aggregated data for presentation.
• Extensively used mongo aggregation framework to aggregate large JSON objects into small objects flattened them to store in RDBMS.
• Independently created Apache NiFi cluster on virtual machine as an alternative to Cloudera Data Flow (CDF).
• Created HA load balancer using Nginx to distribute the load on Apache NiFi.
• Managed NiFi internal and external security using a self-signed and letsencrypt  certificates.
• Created an observability platform for infrastructure and application auditing, logging, monitoring, data quality, and alerting using Prometheus and Grafana.
• Guiding data scientists to take architectural decisions for video analytics framework.
• Coordinated with the product owner, data scientist, and backend/frontend engineers for the fast delivery of the data product.
• Fast pace, agile mindset with hands-on approach for solving problem

• Created a CI/CD pipeline for integration of changes in the operational analytics platform.
• Used multiple technologies (python, shell, git, and Jenkin cluster, bit-bucket repository) to create a data pipeline to move data from many sources (relational database, rest-API, weblogs, streaming platform) to exasol analytics platform.
• Designed Master and Transactional data management layers to capture the business logic using SQL and Lua scripting.
• Utilized python based Meta data driven integration tool m3d-api for data ingestion in data lake and EMR cluster and used m3d-engine for data transformation.
• Designed multiple tools to maintain data consistency and auditing. (Master data reconciliation, Automated Data Quality, Management of Change data capture, Implementation of Global semantic at various regions).
• Created multiple ansible playbooks to create on demand infrastructure in the cloud.
• Proposed scalable analytics platform using D3.js, node.js, Docker and Kubernetes.
• Played role of scrum master ?
  • Preparing user stories with clear and concise descriptions.
  • Hosting daily scrum calls and updating the JIRA, keeping the PO up to date.
  • Planning and executing retrospective, review and backlog grooming meeting, following up on the feedback given by the team.

• Created data pipeline to pull data from Salesforce, Live-ops, Live-Person, Self-service portal, Lithium and Google Analytics API
• Designed a real time streaming pipeline to receive telemetry data from game servers using kinesis events stream.to S3 buckets
• Utilized AWS ? lambda to decompress, transform and enrich raw data into JSON format
• Created data pipeline to move data from s3-bucket to Redshift.

As a Senior Business Intelligence  Developer at Virgin Media in the UK, I was responsible for leading a team of six in the design and development of four data warehouse applications. I used a range of tools and technologies, including Aginity, Toad, SQL Developer, Power center Designer, and IBM Netezza, and programming languages such as SQL and Shell Scripting.

In my role, I worked closely with client SMEs, business users, and owners to plan and implement changes in the application. This involved actively participating in discussions and providing expert guidance on the technical aspects of the project. I also performed impact analysis, created High/Low level design documents, and planned for development and testing to ensure that changes were deployed effectively.

To translate business processes into Informatica mappings and load Star Schema, I utilized my technical expertise in data warehousing and ETL development. I also created automated scripts to perform data cleansing and loading, ensuring that the data was accurate and reliable.

Overall, my work was critical in ensuring that Virgin Media had reliable and effective data warehouse applications to support its business operations. I collaborated closely with other members of the team, including business analysts, designers, and testers, to ensure that the applications met the project requirements and were delivered on time.

• Worked as Team Lead/On-site of BI applications for Virgin media, UK
• Led team of 6 to design and develop 4 Data Warehouse applications.
• Actively involved in discussion with client SME, Business user and owners for planning and implementation of changes in the application.
• Performed impact analysis, created High/Low level design documents, planned for development and testing, deployment of the change in the systems.
• Translated Business processes into Informatica mappings to load Star Schema.
• Created automated scripts to perform data cleansing and data loading
  

• Worked as Team Lead of BI applications for Outokumpu Oyj. (Steel Manufacturer) of Finland.
• Lead team of 5 people to design and develop 3 BI applications.
• Performed data cleansing and loading.
• Designed facts, dimensions and aggregate tables and oracle procedures, packages to load tables in type 1/2.
• Created several KPIs in fact and aggregate tables for reporting.
• Worked as site coordinator and interacted with offshore and client to understand and explain the requirement.
• Performed impact analysis and created High/Low level design documents

• Worked as senior developer of the ETL mappings based on the design provided by the designer.
• Extensively used various transformations in mappings, Workflow, Pl/SQL packages, procedures, materialized view, UNIX scripts to load data from source to target.