PROFESSIONAL EXPERIENCE
Cognizant - Center of Excellence (CoE) Big Data Architect
Technologies: Hadoop stack, Cloudera, Hortonworks
Roles & Responsibilities:
- Playing active role into Big Data projects client acquisition, presenting clients how Big Data solutions can help their business (understanding their struggles and offering solutions)
- Internal networking, connecting with the existing Big Data experts from inside the company (CoE), getting to know about the Cognizant Big Data solutions and success stories details for many customers
GfK ? Lead Big Data Engineer / Big Data Architect
Technologies: Scalding and Spark, HIVE, Cloudera CDH, Data Science & Statistics, Tableau, AWS, S3 / Glacier, MongoDB, MySQL etc.
Roles & Responsibilities:
- Leading Big Data algorithms design and implementations
- Lead the Big Data Experts Hub group, which has the purpose to:
- drive the Big Data adoption inside the company / old & new projects
- guide toward embracing the correct solutions and technologies
- successfully started a set of new Big Data projects meant to replace obsolete / end of life ones
- Significantly optimized several algorithms and flow
- Picked up technologies for performing Proof of Concept + benchmarks + estimations / cost optimizations
- Offered help during critical phases and turned the focus in the right direction to maintain the quality while reducing the efforts
- Designed how to elaborate the Quality Checks for various data levels and offering the KPIs in a simple yet usable way
- Offered technical guidance to internal and external consultants, PM and data scientists / statisticians
- Designed (and implemented parts of) the complex processing flows and data delivery toward GfK client?s presentation software
- Played several roles: technical lead developer, architect for new projects and major changes around ongoing projects
Optymyze ? Big Data Architect / Technical Leader
Technologies: Hadoop stack, MapReduce V2, Cloudera CDH, Cascading, Oracle
Roles & Responsibilities:
- The first employee in the Big Data department, started the first Big Data Proof of Concept in the company, which turned later into a secondary processing capability of the main Oracle based system (mostly used for offloading nightly intensive processing into a Hadoop cluster, storing historical data for very complex computation logic) resulting in a new faster, cost effective processing capability for the Optymyze product, saving the Oracle Enterprise license cost (at least for the customers with huge data volumes; eg. some were running 500+ nightly processing jobs, running for more than 16 hours, drop to 6+ hours on a small Hadoop cluster)
- Built the architecture of a relatively small Hadoop cluster + technologies around it (to store / process the data obtained directly from Oracle / via daily input files / from previous days and months of processing on the Hadoop cluster / from the flow definition and metadata)
- Designed (and implemented some intricate parts of) a dynamically user-definable complex processing flows, involving reusable building blocks based (powered by Cascading)
Amazon ? Lead Big Data Engineer
Technologies: Oracle DWH, Hadoop, Map Reduce, HBase, AWS (20+ Amazon services were used as data sources / destinations: DynamoDB, S3, Clickstream, Shipping services, Payment services, SNS, SQS)
Roles & Responsibilities:
- Led a team of developers during the Sales Dashboard implementation (offloading the Oracle DWHs processing to a new standalone Hadoop Cluster used just for the Seller Central Analytics team) and while revamping the whole set of Business Reports (complex analytical reports for sellers) to use the same cluster, offloading the Oracle DWH and significantly reducing costs (estimated total saving of $1M+ yearly), performing at scale, reliable and much faster than the previous non-Big Data solutions (while also allowing for many new use-cases and complex features which were not possible in older versions)
- Designed the processing flows, identified the faster storage solutions (HBase, DynamoDB, S3), performed benchmarks and Proof of Concepts, imagined architecture options and drove the technical discussions; acted as key-contact person toward management upper layer;
- This solution was reused later to create a sort of Reporting Marketplace as a Service
Other companies (telecommunication) ? Manager / Team Leader
Technologies: Java, Oracle, various web frontend technologies
Roles & Responsibilities:
- Managed several teams of developers (9+ years)
- Designed the software architecture, high and low level design for various projects and feature additions
- Drove technical discussions