a Randstad company

(Senior) Data Engineer mit Schwerpunkt auf data-intensive Pipelines in Azure, Python, PySpark und Databricks

Profil
Top-Skills
Python PySpark Azure SQL Shell Databricks Scrum CICD Git Clean Code ELK Stack Airflow PyTest NoSQL flask pandas numpy scikit-learn cloud architectures
Verfügbar ab
22.09.2022
Aktuell verfügbar - Der Experte steht für neue Projektangebote zur Verfügung.
Verfügbar zu
100%
davon vor Ort
100%
Einsatzorte

PLZ-Gebiete
Länder
Ganz Deutschland, Österreich, Schweiz
Remote-Arbeit
möglich
Art des Profiles
Freiberufler / Selbstständiger
Der Experte ist als Einzelperson freiberuflich oder selbstständig tätig.

9 Monate

2022-01

heute

Data Integration Platform

Senior Data Engineer Python Azure Databricks (Delta Lake) ...
Rolle
Senior Data Engineer
Projektinhalte

  • Part of agile team (?30 members) providing cloud data platform for industry-leading business departments. 
  • Designed and implemented architecture for ?30 ETL applications and APIs processing global IoT data on a terabyte scale. Deployment of various managed services in hybrid cloud setup. 
  • Enablement of client developers. Optimized storage to save ?4000? per month.

Kenntnisse
Python Azure Databricks (Delta Lake) PySpark SQL NiFi Shell CI/CD Terraform
Kunde
inovex
Einsatzort
Munich
1 Jahr 4 Monate

2020-09

2021-12

ETL & Dashboarding on sensitive IoT data

Data Engineer Python Azure Elasticsearch ...
Rolle
Data Engineer
Projektinhalte

  • Designed and implemented architecture for ETL and dashboarding application for IoT data. 
  • High focus on security for sensitive laboratory data on Azure. 
  • Realized clean software architecture principles for maintainability of 11 Python projects. 
  • Reduced processing time by a factor of ?50. 
  • Conducted requirements engineering as main contact for stakeholders.

Kenntnisse
Python Azure Elasticsearch Kibana Shell CI/CD
Kunde
inovex
Einsatzort
Munich
8 Monate

2021-02

2021-09

Data Lake & Predictive Maintenance

Data Engineer Python Azure Databricks (Delta Lake) ...
Rolle
Data Engineer
Projektinhalte

  • Part of agile team (6 members) with goal to provide data-driven insights to end-users on globally distributed IoT data of industry-leading client. 
  • Designed and implemented architecture for ETL and machine learning applications that process data daily on a giga- and terabyte scale. 
  • Full responsibility from raw data collection over processing with machine learning to visualization.
  • Deployment of various managed services in private network on Azure. 
  • Enablement of client developers.

Kenntnisse
Python Azure Databricks (Delta Lake) PySpark SQL Shell CI/CD Terraform
Kunde
inovex
Einsatzort
Munich
4 Monate

2020-10

2021-01

Cloud Migration to Azure

Cloud Engineer Azure Terraform Azure DevOps ...
Rolle
Cloud Engineer
Projektinhalte

Migrated ETL pipelines and web APIs from on-premises to Docker-based services on Azure.

Kenntnisse
Azure Terraform Azure DevOps CI/CD ARM templates Docker
Kunde
inovex
Einsatzort
Munich.
4 Monate

2020-06

2020-09

ETL & Predictive Maintenance Web-Service

Machine Learning Engineer Python Shell Gitlab CI/CD ...
Rolle
Machine Learning Engineer
Projektinhalte

Designed and implemented on-premises ETL pipelines and web APIs for ML applications.

Kenntnisse
Python Shell Gitlab CI/CD Elasticsearch Kibana flask
Kunde
inovex
Einsatzort
Munich
1 Jahr 4 Monate

2019-02

2020-05

Evaluated machine learning frameworks

Data Scientist, Student extensive Python programming scikit-learn TensorFlow ...
Rolle
Data Scientist, Student
Projektinhalte
Evaluated machine learning frameworks with focus on tabular data sets.
Kenntnisse
extensive Python programming scikit-learn TensorFlow PyTorch
Kunde
inovex GmbH
Einsatzort
Munich
8 Monate

2019-09

2020-04

Designed and realized data collection pipeline

Guest Researcher IoT Python scikit-learn ...
Rolle
Guest Researcher
Projektinhalte

  • Designed and realized data collection pipeline for thermal comfort related data including biosignals and occupant controls. 
  • Optimized, deployed and evaluated machine learning models for infrared heating panel control with focus on interpretability. 
  • Reduced command frequency by 79% and increased thermal comfort to 89%. 
  • Published findings (paper, code, data).

Kenntnisse
IoT Python scikit-learn fastai Swift Cloud Computing Expainable AI
Kunde
Carnegie Mellon University
Einsatzort
Pittsburgh
4 Monate

2018-10

2019-01

Evaluated distributed database systems

Data Engineer, Student self-responsible working Kudu Hadoop ...
Rolle
Data Engineer, Student
Projektinhalte
Evaluated distributed database systems in Hadoop landscape.
Kenntnisse
self-responsible working Kudu Hadoop HBase Parquet
Kunde
inovex GmbH
Einsatzort
Munich
5 Monate

2018-03

2018-07

Designed and implemented machine learning model

ML Engineer, Student Python scikit-learn AWS ...
Rolle
ML Engineer, Student
Projektinhalte
Designed and implemented dough kneading machine learning model in agile team. Improved productivity by at least 30%.
Kenntnisse
Python scikit-learn AWS Swift CI/CD presenting in front of hundreds
Kunde
DIOSNA
Einsatzort
Munich

7 Monate

2020-07

2021-01

Part-time advanced training - Data Engineering Nanodegree

Udacity, online
Institution, Ort
Udacity, online
Schwerpunkt
Skills: AWS, PySpark, Airflow, Python, Shell, SQL, NoSQL, data modeling
3 Jahre 7 Monate

2016-10

2020-04

Informatics

M.Sc., Technical University Munich
Abschluss
M.Sc.
Institution, Ort
Technical University Munich
Schwerpunkt

software engineering & architecture, agile project management, machine learning, efficient algorithms


1 Jahr

2018-10

2019-09

Data Engineering & Analytics

M.Sc. - cancelled, Technical University Munich
Abschluss
M.Sc. - cancelled
Institution, Ort
Technical University Munich
Schwerpunkt

  • data engineering, machine learning, distributed systems, deep learning
  • Grade when cancelled: 1.9

6 Monate

2017-10

2018-03

Study Abroad

Multimedia University Malaysia
Institution, Ort
Multimedia University Malaysia
Schwerpunkt
software engineering, software quality
3 Jahre 1 Monat

2013-10

2016-10

Informatics

B.Sc., Technical University Munich
Abschluss
B.Sc.
Institution, Ort
Technical University Munich
Schwerpunkt

  • software engineering, machine learning, business analytics, database systems
  • Final grade: 2.3

Data Engineer

Deutsch Muttersprache
Englisch Verhandlungssicher

Top Skills
Python PySpark Azure SQL Shell Databricks Scrum CICD Git Clean Code ELK Stack Airflow PyTest NoSQL flask pandas numpy scikit-learn cloud architectures
Produkte / Standards / Erfahrungen / Methoden
Coding:

Expert: Python, SQL, Shell 


Extended:

JavaScript, Java, C# Basic: R, Scala, C++, Swift Azure Data Factory, Functions, App Services, Container Instances, DevOps, Key Vault, Event Grid, MS SQL, Logic Apps, Blob, ADLS, Queues, Fileshare 


AWS 

EC2, S3, Redshift, Elastic Map Reduce 


Others:

Databricks, PySpark, CI/CD, Scrum, Git, Clean Code, NumPy, pandas, Bash, NoSQL, Jupyter, RDBMS, Elasticsearch, Kibana, PyTest, client-facing communication, Terraform, Docker, Airflow, Data Mesh, scikit-learn, matplotlib, MongoDB, Cassandra, influxDB, Atlassian Stack, UML, flask, sparklyr, TensorFlow, HDFS, PyTorch, Explainable AI

Awards gladly on request
Ihr Kontakt zu Gulp

Fragen? Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Jetzt bei GULP Direkt registrieren und Freelancer kontaktieren