Data Engineer mit Erfahrung in Cloud und onPrem best practices
Aktualisiert am 02.07.2024
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 01.08.2024
Verfügbar zu: 100%
davon vor Ort: 100%
Kotlin
Java
Python
Back-End
Apache Spark
Spring
Kafka
Big Query
GCP
Pubsub
Spring boot
Big Table
AlloyDB
Arduino
C++
Arabic
Mother Tongue
English
Fluent
German
Fluent

Einsatzorte

Einsatzorte

Hamburg (+20km) München (+20km) Berlin (+20km) Frankfurt am Main (+20km) Köln (+20km)
Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

3 Jahre 9 Monate
2021-04 - heute

Java Software Developer

Java Software Developer Java Python Kotlin ...
Java Software Developer
Spring Kafka PostgreSQL AWS Big Query Big Table Pubsub Spring boot Good Clinical Practice
Java Python Kotlin Arduino
9 Jahre
2016-01 - heute

Management

Co-Founder Angularjs Postgresql Google Pub/Sub ...
Co-Founder


Angularjs Postgresql Google Pub/Sub Google Data Store Google Cloud Storage Java 11 Spring Boot Kotlin Android
on request
1 Jahr 6 Monate
2019-10 - 2021-03

Android Developer

Android Developer Java Kotlin Android
Android Developer

Android Entwicklung, Java Spring Boot, Kotlin

Prometheus Grafana Android Room Retrofit Hilt
Java Kotlin Android
Hamburg
9 Monate
2018-12 - 2019-08

Architecture design & development plan

Head of Engineering, Banks Mgmt & Standard Reporting Kotlin Java Apache Spark ...
Head of Engineering, Banks Mgmt & Standard Reporting
  • Build team an Agile team
  • Disciplinary & Professional leadership of team
  • Plan Development Cycles with the team and moderate the sessions
  • Architecture design & development plan
  • Build Standard Reporting Platform
  • Manage Integration platform Team
  • Build CI/CD Pipelines
  • Build ETL service using Spark
  • Build Data Quality Service
Kotlin Java Apache Spark Spring Boot Apache Camel PostgreSQL Snow Flake Amazon S3 Docker Jenkins Kubernetes Grafana Prometheus
Deposit Solutions GmbH, Hamburg
3 Monate
2018-11 - 2019-01

Data Migration

Data Engineer Apache NiFi Scrum Hadoop ...
Data Engineer

DWH Star Schema Design und aufbauen und die ETL prozesse dazu einsetzen

Spring Hive Hortonworks
Apache NiFi Scrum Hadoop Hive Data Engineering Java
München
1 Monat
2018-11 - 2018-11

Design & Build

Senior Data Engineer Apache Hive NiFi Hortonworks Data Platform
Senior Data Engineer
  • Design & Build ETL Processes with NiFi
  • Build BI-Data Warehouse Staging Layer
Apache Hive NiFi Hortonworks Data Platform
Telefónica Germany GmbH & Co. OHG, München
1 Jahr 9 Monate
2017-02 - 2018-10

Concepting the Architecture for a Platform

Senior Platform Architect/PO Scrum Scala Batch-Streaming ...
Senior Platform Architect/PO
  • Concepting the Architecture for a Platform as a Service for data science
  • Build a cross functional team , and plan the resources and competences
  • Plan Development Cycles with the team and moderate the sessions
  • Sprint Planning and Tickets Grooming
  • Architectural Design and Communication Protocols
  • Automated Model Training and Deployment
  • Develop Batch Stream Processing System
  • Develop Kafka Data Aggregation Pattern
  • Apache NiFi Data Flow Management
  • Real-TIme Data Processing
  • Data Monitoring & Alerting Systems
  • Manage GDPR compliance project
HBase Hive Kafka Exadata Openshift Hortonworks HDP
Scrum Scala Batch-Streaming Stream Processing Machine Learning as a Service Python
Hamburg
7 Monate
2016-06 - 2016-12

System Design & Architecture

Senior Development Consultant Mesos Docker Chronos ...
Senior Development Consultant
  • Successfully led the customer’s team to build the system on time and budget
  • Big Data Processing Platform, and processes scheduler
  • System Design & Architecture
  • Build a Microservices platform
    • Jobs Scheduler
    • Jobs Dependency Graphs Resolution
    • SDK & Cockpit
    • Resources Calculator - Machine Learning
    • Monitoring, management and clean up services (Scala Services)
  • Build RealTime jobs as well as scheduled jobs, Lambda Architecture
  • Build a data consolidation and management platform of services
Mesos Docker Chronos Marathon Scala Python R Hadoop Hive HBase RabbitMQ SaltStack
Data Drivers GmbH, Hamburg/ Kreditech Holding SSL GmbH
1 Jahr 7 Monate
2014-05 - 2015-11

web developers

Head of R&D Python Scala JS & Java ...
Head of R&D
  • Led a team web developers to get the first version of the website online in 3 months
  • Led a team of 3 Data Engineers to build a products’ matching system
  • Led a team of 3 Data Engineers to build a products’ classification system based on text information
  • Systems Built:
    • Real-Time Image processing & caching
    • Semantics-Analysis and classification system
    • Web Crawler for products info and specs collection from the web
  • Product Features from Images extraction system
  • Product Features from Texts extraction system
Python Scala JS & Java Apache Spark Scikit-Learn Hadoop OpenCV & NLTK Docker App-Engine Google Pub/Sub & Google Data Storage Postgresql Google BigQuery Data Storage & OrientDB
SNM Style Net Media GmbH, Hamburg
1 Jahr 1 Monat
2013-04 - 2014-04

Planned & designed

SOA Team Leader PHP Python MSSQL ...
SOA Team Leader
  • Led a team of 4 web developers to build a SOA for the webshop
  • Planned & designed the system with the CTO
  • Chose the technologies and supervised the system building process
  • Design and build the PHP CMS
  • Develop the company’s DevOps processes and operations
PHP Python MSSQL PostgreSQL
HQ Patronen GmbH, Hamburg
6 Monate
2012-01 - 2012-06

Website Administration

IT Manager PHP Shell Scripting Networking
IT Manager
  • Build the company’s IT infrastructure
  • Do the DevOps processes
  • Website Administration, management and development
PHP Shell Scripting Networking
Agenture Gretchen, Berlin
4 Jahre 4 Monate
2007-09 - 2011-12

Manage IT Projects

Technical Project Manager ASP.NET C# MSSQL
Technical Project Manager
  • Manage IT Projects with outsourced teams in India
  • Develop Systems and Prototypes for financial analysts to manage the budget
  • Develop department internal website in ASP.NET, C# and MS-SQL
  • Develop algorithms to reduce expenses and circuits lengths
ASP.NET C# MSSQL
Orange Business Services, Cairo

Aus- und Weiterbildung

Aus- und Weiterbildung

7/2005

Study: B.Sc. Of Computer Science & Information Systems
Institute: 6 October University, Cairo, Egypt

Kompetenzen

Kompetenzen

Top-Skills

Kotlin Java Python Back-End Apache Spark Spring Kafka Big Query GCP Pubsub Spring boot Big Table AlloyDB Arduino C++

Produkte / Standards / Erfahrungen / Methoden

Apache Flink
Apache Hadoop
Apache NiFi
Hadoop
HDP
Hive
Hortonworks
Kafka
Machine Learning as a Service
Openshift
Scrum
Spring
C++
Big Query
Big Table
GCP
Vertex Ai
Alloy DB
Arduino
Open AI
Postgresql
Mysql

Skills

  • Scrum, Kanban, Project Management, Leadership
  • Microservices, Lambda, Kappa, Event-Driven & SOA Architectures
  • Python, Scala, Java, R & Javascript
  • Hadoop, Cassandra, Hive, Mesos, HBase, Mesos Frameworks & Apache Spark
  • SQL, Postgresql, Microsoft SQL & Mysql
  • Google Cloud Platform, Amazon AWS
  • CI, Docker, Vagrant, Ansible, Bower & Grunt

Project Management

  • Agile Leadership
  • Product Owner/Manager


System Architecture

  • Lambda
  • Kappa
  • Event Driven
  • Microservices
  • SOA

Apache

  • Hadoop, Hive, HBase, Cassandra, Kafka, Ranger, Atlas
  • Spark, Storm, Solr, Lucene, Mesos

CI

  • Jenkins
  • Docker
  • Marathon
  • Chronos
  • Kubernetes
  • Openshift

Cloud

  • Google Cloud Platform
  • Amazon AWS

DevOp

  • Vagrant
  • Ansible
  • Puppet
  • RedHat Satelite
  • FreeIPA

Programmiersprachen

Java
Kotlin
Python

Datenbanken

Apache Cassandra
Exadata
HBase
Hive
MongoDB
MS-SQL
MySQL
Neo4j
Oracle
OrientDB
PostgreSQL
Big Query
Big Table
AlloyDB
Firebase

Design / Entwicklung / Konstruktion

Architecture
Batch-Streaming
Data Engineering
Microservices
Stream Processing

Einsatzorte

Einsatzorte

Hamburg (+20km) München (+20km) Berlin (+20km) Frankfurt am Main (+20km) Köln (+20km)
Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

3 Jahre 9 Monate
2021-04 - heute

Java Software Developer

Java Software Developer Java Python Kotlin ...
Java Software Developer
Spring Kafka PostgreSQL AWS Big Query Big Table Pubsub Spring boot Good Clinical Practice
Java Python Kotlin Arduino
9 Jahre
2016-01 - heute

Management

Co-Founder Angularjs Postgresql Google Pub/Sub ...
Co-Founder


Angularjs Postgresql Google Pub/Sub Google Data Store Google Cloud Storage Java 11 Spring Boot Kotlin Android
on request
1 Jahr 6 Monate
2019-10 - 2021-03

Android Developer

Android Developer Java Kotlin Android
Android Developer

Android Entwicklung, Java Spring Boot, Kotlin

Prometheus Grafana Android Room Retrofit Hilt
Java Kotlin Android
Hamburg
9 Monate
2018-12 - 2019-08

Architecture design & development plan

Head of Engineering, Banks Mgmt & Standard Reporting Kotlin Java Apache Spark ...
Head of Engineering, Banks Mgmt & Standard Reporting
  • Build team an Agile team
  • Disciplinary & Professional leadership of team
  • Plan Development Cycles with the team and moderate the sessions
  • Architecture design & development plan
  • Build Standard Reporting Platform
  • Manage Integration platform Team
  • Build CI/CD Pipelines
  • Build ETL service using Spark
  • Build Data Quality Service
Kotlin Java Apache Spark Spring Boot Apache Camel PostgreSQL Snow Flake Amazon S3 Docker Jenkins Kubernetes Grafana Prometheus
Deposit Solutions GmbH, Hamburg
3 Monate
2018-11 - 2019-01

Data Migration

Data Engineer Apache NiFi Scrum Hadoop ...
Data Engineer

DWH Star Schema Design und aufbauen und die ETL prozesse dazu einsetzen

Spring Hive Hortonworks
Apache NiFi Scrum Hadoop Hive Data Engineering Java
München
1 Monat
2018-11 - 2018-11

Design & Build

Senior Data Engineer Apache Hive NiFi Hortonworks Data Platform
Senior Data Engineer
  • Design & Build ETL Processes with NiFi
  • Build BI-Data Warehouse Staging Layer
Apache Hive NiFi Hortonworks Data Platform
Telefónica Germany GmbH & Co. OHG, München
1 Jahr 9 Monate
2017-02 - 2018-10

Concepting the Architecture for a Platform

Senior Platform Architect/PO Scrum Scala Batch-Streaming ...
Senior Platform Architect/PO
  • Concepting the Architecture for a Platform as a Service for data science
  • Build a cross functional team , and plan the resources and competences
  • Plan Development Cycles with the team and moderate the sessions
  • Sprint Planning and Tickets Grooming
  • Architectural Design and Communication Protocols
  • Automated Model Training and Deployment
  • Develop Batch Stream Processing System
  • Develop Kafka Data Aggregation Pattern
  • Apache NiFi Data Flow Management
  • Real-TIme Data Processing
  • Data Monitoring & Alerting Systems
  • Manage GDPR compliance project
HBase Hive Kafka Exadata Openshift Hortonworks HDP
Scrum Scala Batch-Streaming Stream Processing Machine Learning as a Service Python
Hamburg
7 Monate
2016-06 - 2016-12

System Design & Architecture

Senior Development Consultant Mesos Docker Chronos ...
Senior Development Consultant
  • Successfully led the customer’s team to build the system on time and budget
  • Big Data Processing Platform, and processes scheduler
  • System Design & Architecture
  • Build a Microservices platform
    • Jobs Scheduler
    • Jobs Dependency Graphs Resolution
    • SDK & Cockpit
    • Resources Calculator - Machine Learning
    • Monitoring, management and clean up services (Scala Services)
  • Build RealTime jobs as well as scheduled jobs, Lambda Architecture
  • Build a data consolidation and management platform of services
Mesos Docker Chronos Marathon Scala Python R Hadoop Hive HBase RabbitMQ SaltStack
Data Drivers GmbH, Hamburg/ Kreditech Holding SSL GmbH
1 Jahr 7 Monate
2014-05 - 2015-11

web developers

Head of R&D Python Scala JS & Java ...
Head of R&D
  • Led a team web developers to get the first version of the website online in 3 months
  • Led a team of 3 Data Engineers to build a products’ matching system
  • Led a team of 3 Data Engineers to build a products’ classification system based on text information
  • Systems Built:
    • Real-Time Image processing & caching
    • Semantics-Analysis and classification system
    • Web Crawler for products info and specs collection from the web
  • Product Features from Images extraction system
  • Product Features from Texts extraction system
Python Scala JS & Java Apache Spark Scikit-Learn Hadoop OpenCV & NLTK Docker App-Engine Google Pub/Sub & Google Data Storage Postgresql Google BigQuery Data Storage & OrientDB
SNM Style Net Media GmbH, Hamburg
1 Jahr 1 Monat
2013-04 - 2014-04

Planned & designed

SOA Team Leader PHP Python MSSQL ...
SOA Team Leader
  • Led a team of 4 web developers to build a SOA for the webshop
  • Planned & designed the system with the CTO
  • Chose the technologies and supervised the system building process
  • Design and build the PHP CMS
  • Develop the company’s DevOps processes and operations
PHP Python MSSQL PostgreSQL
HQ Patronen GmbH, Hamburg
6 Monate
2012-01 - 2012-06

Website Administration

IT Manager PHP Shell Scripting Networking
IT Manager
  • Build the company’s IT infrastructure
  • Do the DevOps processes
  • Website Administration, management and development
PHP Shell Scripting Networking
Agenture Gretchen, Berlin
4 Jahre 4 Monate
2007-09 - 2011-12

Manage IT Projects

Technical Project Manager ASP.NET C# MSSQL
Technical Project Manager
  • Manage IT Projects with outsourced teams in India
  • Develop Systems and Prototypes for financial analysts to manage the budget
  • Develop department internal website in ASP.NET, C# and MS-SQL
  • Develop algorithms to reduce expenses and circuits lengths
ASP.NET C# MSSQL
Orange Business Services, Cairo

Aus- und Weiterbildung

Aus- und Weiterbildung

7/2005

Study: B.Sc. Of Computer Science & Information Systems
Institute: 6 October University, Cairo, Egypt

Kompetenzen

Kompetenzen

Top-Skills

Kotlin Java Python Back-End Apache Spark Spring Kafka Big Query GCP Pubsub Spring boot Big Table AlloyDB Arduino C++

Produkte / Standards / Erfahrungen / Methoden

Apache Flink
Apache Hadoop
Apache NiFi
Hadoop
HDP
Hive
Hortonworks
Kafka
Machine Learning as a Service
Openshift
Scrum
Spring
C++
Big Query
Big Table
GCP
Vertex Ai
Alloy DB
Arduino
Open AI
Postgresql
Mysql

Skills

  • Scrum, Kanban, Project Management, Leadership
  • Microservices, Lambda, Kappa, Event-Driven & SOA Architectures
  • Python, Scala, Java, R & Javascript
  • Hadoop, Cassandra, Hive, Mesos, HBase, Mesos Frameworks & Apache Spark
  • SQL, Postgresql, Microsoft SQL & Mysql
  • Google Cloud Platform, Amazon AWS
  • CI, Docker, Vagrant, Ansible, Bower & Grunt

Project Management

  • Agile Leadership
  • Product Owner/Manager


System Architecture

  • Lambda
  • Kappa
  • Event Driven
  • Microservices
  • SOA

Apache

  • Hadoop, Hive, HBase, Cassandra, Kafka, Ranger, Atlas
  • Spark, Storm, Solr, Lucene, Mesos

CI

  • Jenkins
  • Docker
  • Marathon
  • Chronos
  • Kubernetes
  • Openshift

Cloud

  • Google Cloud Platform
  • Amazon AWS

DevOp

  • Vagrant
  • Ansible
  • Puppet
  • RedHat Satelite
  • FreeIPA

Programmiersprachen

Java
Kotlin
Python

Datenbanken

Apache Cassandra
Exadata
HBase
Hive
MongoDB
MS-SQL
MySQL
Neo4j
Oracle
OrientDB
PostgreSQL
Big Query
Big Table
AlloyDB
Firebase

Design / Entwicklung / Konstruktion

Architecture
Batch-Streaming
Data Engineering
Microservices
Stream Processing

Vertrauen Sie auf Randstad

Im Bereich Freelancing
Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

Fragen?

Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Das Freelancer-Portal

Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.