a Randstad company

envelope

+49 89 500316-300

Jetzt registrieren

Cloud data migration, Data Engineering, Hadoop, Big Data, MapR, HPE Ezmeral Data Fabric, Kafka, Cassandra, JupyterHub, Analytics Tools.

Profil
Top-Skills
Hadoop eco-system managements Cloud data migration Big Dig System Engineer. Hadoop eco-system MapR eco-system GCP Spark Hive Airflow JupyterHub Dataproc
Verfügbar ab
01.01.2023
Noch verfügbar - Schnell sein lohnt sich: Der Experte kann bereits für Projekte vorgesehen sein.
Verfügbar zu
100%
davon vor Ort
100%
Einsatzorte
PLZ-Gebiete
Länder
Ganz Deutschland, Österreich, Schweiz
Remote-Arbeit
möglich
Art des Profiles
Freiberufler / Selbstständiger
Der Experte ist als Einzelperson freiberuflich oder selbstständig tätig.

5 Jahre 4 Monate

2017-10

heute

Leading cross-functional team

Technical team lead DATA Ops
Rolle
Technical team lead DATA Ops
Projektinhalte

  • Leading cross-functional team including OPS and Dev members 
  • Part of core team for DWH (Oracle, Hadoop) migration to GCP 
  • Design and setup of GCP projects via Terraform (GCS, BigQuery, Dataproc, Dataflow, Pub/Sup, Cloud Composer, Cloud Build, etc.) 
  • Evaluation of Databricks platform including Delta Live Table, Job orchestration, Databricks SQL, Cluster policies, Dashboards and monitoring and alerting 
  • Design and implementation of PII data pseudonymization for cloud migration 
  • Installation, configuration and maintenance of MapR eco-system, fully automated by Ansible and Puppet (MapR core, Spark, Hive, Hue, Livy, Objectstore, NFS and monitoring packs) 
  • Setup and configuration on MapR Monitoring solution (OpenTSDB, Elasticsearch, CollectD, FluentD, Kibana, Grafana) 
  • Major upgrade and patching of MapR clusters with full automation by Ansible 
  • MapR cross-clusters data replication setup, cross cluster data access setup 
  • Containerization of MapR client/edge node by Ansible automation (OpenShift, Podman, Buildah, Ansible bender)
  • Datacenter migration planning and execution which include migration of all MapR clusters to new DC 
  • Data restructuring to achieve better Application (Spark, Hive) performance
  • Application tuning (SQL performance tuning) for Spark and Hive 
  • Spark streaming application development to replace Apache Flume 
  • Apache Airflow installation, configuration, update and maintenance on OpenShift. Application orchestration by Airflow 
  • JupyterHub installation and configuration including custom configuration per user 
  • Installation and configuration of Datahub/OpenMetadata (Data Catalog) including inhouse development of data lineage generator for different data sources (Oracle, Hive, SAS) 
  • Installation, configuration and maintenance of Cassandra, Kafka, Postgres and Redis 
  • PoCs on different in-memory query engines (Apache Drill, Trino, Starburst)

Produkte
Oracle Hadoop GCS BigQuery Dataproc Dataflow Pub/Sup Cloud Composer Cloud Build MapR core Spark Hive Hue Livy Objectstore NFS monitoring packs OpenShift Podman Buildah Ansible bender SAS Cassandra Kafka Postgres and Redis Apache Drill Trino Starburst
Kunde
PAYBACK GmbH
Einsatzort
Munich, Germany
1 Jahr 9 Monate

2020-01

2021-09

MapR cross-clusters replication setup

BIGDATA Specialist
Rolle
BIGDATA Specialist
Projektinhalte
  • MapR cross-clusters replication setup, automated with Ansible. 
  • MapR cross clusters data access setup. 
  • Containerization of MapR client/edge node via Ansible automation. 
  • Monitoring application performance on MapR eco-system (Hadoop cluster). 
  • Tunning MapR cluster to maximize utilization. 
  • PoCs on different in-memory query engines (Apache Drill, Trino, Starburst). 
  • Setup and configure application orchestration with Apache Airflow. 
  • JupyterHub installation and configuration.
Kunde
PAYBACK GmbH
Einsatzort
Munich, Germany
2 Jahre 4 Monate

2017-10

2020-01

Migration of multiple MapR (Hadoop) clusters

BIGDATA system engineer
Rolle
BIGDATA system engineer
Projektinhalte
  • Migration of multiple MapR (Hadoop) clusters between datacenters. 
  • Migration of Data and Applications between clusters. 
  • Installation, configuration and maintenance of MapR eco-system, fully automated by Ansible and Puppet. 
  • Installation, configuration and maintenance of Cassandra, Kafka, Postgres and Redis.
  • Setup and configuration of monitoring tools (OEM and Icinga).
Kunde
PAYBACK GmbH
Einsatzort
Munich, Germany
5 Jahre

2012-10

2017-09

Delegate duties and tasks

Team lead IT Operations
Rolle
Team lead IT Operations
Projektinhalte
  • Delegate duties and tasks within the IT department 
  • Perform regular IT audit to discover areas of weaknesses and fortify them 
  • OS installation, configuration, troubleshooting and tuning for Linux Red Hat 6 and 7 with security enhancement through SELINUX
  • Configuration and administration of HA Postgres cluster with backup and recovery validation 
  • Installation and configuration of load balancer and caching mechanism (Varnish) and web servers (Apache, Nginx) 
  • Setup and configuration of monitoring tools (Check_mk, Graphite, Grafana). Log management and analysis (Elasticsearch, Kibana, FluentD) 
  • Configuration and troubleshooting of various network components (Juniper and Cisco) and implementation of Policy rules, DMZ and multi cross Vlan communications 
  • User management and authentication via LDAP and Radius 
  • Infrastructure automation with Ansible
Produkte
Varnish Apache Nginx Check_mk Graphite Grafana Elasticsearch Kibana FluentD Juniper and Cisco
Kunde
Xyrality GmbH
Einsatzort
Hamburg, Germany
2 Jahre 8 Monate

2012-11

2015-06

installation, configuration, troubleshooting and tuning

Linux System Administrator
Rolle
Linux System Administrator
Projektinhalte
  • OS installation, configuration, troubleshooting and tuning for Linux Red Hat 6 and 7 with security enhancement through SELINUX. 
  • Configuration and administration of HA Postgres cluster with backup and recovery validation. 
  • Installation and configuration of load balancer and caching mechanism (Varnish) and web servers (Apache, Nginx). 
  • Setup and configuration of monitoring tools (Check_mk, Graphite, Grafana). Log management and analysis (ELK tools). 
  • Configuration and troubleshooting of various network components (Juniper and Cisco) and implementation of Policy rules, DMZ and multi cross Vlan communications. 
  • User management and authentication via LDAP and RADIUS. 
  • Infrastructure automation with Ansible.  
Kunde
Xyrality GmbH
Einsatzort
Hamburg, Germany
1 Jahr 1 Monat

2011-09

2012-09

Setup and maintenance of High-Performance Computing cluster

Research Assistant at Faculty of Dentistry (Linux System Administrator)
Rolle
Research Assistant at Faculty of Dentistry (Linux System Administrator)
Projektinhalte
  • Setup and maintenance of High-Performance Computing cluster with Rocks. 
  • Integration of servers to Cluster machine to be used for Bio-Informatics research. 
  • Server Virtualization by KVM and RHEV. 
  • Install and manage various Bio-Informatics software.
Kunde
University of Malaya
Einsatzort
Kuala Lumpur, Malaysia
2 Jahre 7 Monate

2009-02

2011-08

Troubleshooting servers

2nd level support engineer (Linux/Windows)
Rolle
2nd level support engineer (Linux/Windows)
Projektinhalte
  • Troubleshooting servers, different kind of Operating Systems (Linux and Windows servers - Windows Server 2003, 2008). 
  • Troubleshoot various problems ranging from control panel, OS, software, applications, emails, database, DNS and many more. 
  • Configuring mail servers. 
  • Assemble and change of hardware for servers. 
  • Being shift coordinator and distributing tasks to team members. 
  • Monitoring servers and network via Monitoring tools such as Nagios and MRTG graphs.
Kunde
Shinjiru Technology Sdn Bhd
Einsatzort
Kuala Lumpur, Malaysia
3 Monate

2010-03

2010-05

Installing and configuring server

Service Delivery Team Member
Rolle
Service Delivery Team Member
Projektinhalte
  • Installing and configuring server with different platform (Windows and Linux). 
  • Troubleshooting servers for any hardware failure. 
  • Service and maintain microcomputer system, including test, run Scandisk, defragment hard drives, update antiviral programs to maintain performance and security
Kunde
Shinjiru Technology Sdn Bhd
Einsatzort
Kuala Lumpur, Malaysia
5 Monate

2008-09

2009-01

Developing password less authentication system

Research Assistant at Faculty of Computer Science
Rolle
Research Assistant at Faculty of Computer Science
Projektinhalte
  • Developing password less authentication system to ease client-server communication on Mobile device. 
  • Synchronization of self-login authentication among the paired devices.
Kunde
University of Malaya
Einsatzort
Kuala Lumpur, Malaysia
7 Monate

2007-01

2007-07

self-support portal

Software Developer (Internship)
Rolle
Software Developer (Internship)
Projektinhalte
  • Part of a team developing a self-support portal for the employee of Deutsch Bank, Australia. 
  • The portal was built using ASP and later was deployed in Deutsch Bank, Melbourne, Australia.  
Kunde
TATA Consultancy Services Ltd
Einsatzort
Bangalore, India

2003 - 2007
Bachelor of Engineering, Information Science and Engineering
Visvesvaraya Technological University, India

Certifications:
  • Google Cloud Platform
    • Associate Cloud Engineer
  • Databricks
    • Certified Data Engineer Associate
  • Red Hat
    • Certified System Administrator (RHCSA), Red Hat Licens
  • IBM
    • Certified Solution Advisor - Cloud Computing Architecture V1

Farsi Native
English Fluent
German Intermediate

Top Skills
Hadoop eco-system managements Cloud data migration Big Dig System Engineer. Hadoop eco-system MapR eco-system GCP Spark Hive Airflow JupyterHub Dataproc
Produkte / Standards / Erfahrungen / Methoden

Profile

Senior Big Data engineer with 10 years of management and architecture design experience. Five years of Bigdata mission critical system management experience in the financial sector (data engineering and system engineering). Technical manager of the operations and evolution of a several Petabyte size Hadoop environments (several clusters, with individual clusters bigger than 1PB stored data and 100s of daily applications). Managing a team of 8 full time data and system engineers responsible for incident followup and operations support coordination (cluster management and application troubleshooting). Responsible architect for on-premise to cloud migration, evaluating GCP cloud technology for Big Data.

Ihr Kontakt zu Gulp

Fragen? Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Jetzt bei GULP Direkt registrieren und Freelancer kontaktieren