Data Engineer
Aktualisiert am 15.09.2026
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 15.09.2026
Verfügbar zu: 100%
davon vor Ort: 100%
Python
SQL
Machine Learning
Pyspark
AWS
Azure
Databricks
dbt
ETL
Apache Cassandra
Apache Airflow
GCP
Stakeholdermanagement
MS Power BI
Kubernetes
Terraform
GenAI
PostgreSQL

Einsatzorte

Einsatzorte

Deutschland
möglich

Projekte

Projekte

3 months
2026-01 - 2026-03

Real-Time Retail Data Pipeline

Kafka PySpark Airflow ...
  • Processed 50 events per second of continuous, timestamped streaming data, as measured by throughput across 6 Kafka topics, by architecting a real-time ingestion pipeline with PySpark and Airflow orchestration into a PostgreSQL-backed data lake.
  • Achieved a 100/100 data quality score, as measured by automated validation coverage, by implementing 19 dbt tests across transformation layers handling high-frequency time-stamped records.
  • Cut operational overhead by 40%, as measured by reduced manual workflow effort, by containerising the full pipeline with Docker and a CI/CD deployment process.
  • Reduced end-to-end ingestion-to-availability latency to 38 seconds, as measured by pipeline throughput timing, by tuning PySpark transformation and Airflow scheduling logic for continuous data arrival.
Kafka PySpark Airflow dbt PostgreSQL MinIO Docker CI/CD
5 months
2025-09 - 2026-01

LakeCatalog-AI

Kubernetes Helm Kustomize ...
  • Sustained 99%+ platform uptime, as measured by automated deployment reliability, by architecting a Kubernetes GitOps pipeline (Helm/Kustomize/ArgoCD) hosting a Databricks-backed metadata Q&A agent.
  • Reduced compute costs by 35%, as measured by resource efficiency, by applying domain-oriented Data Mesh design principles.
  • Enabled fast, self-service data querying, as measured by dimensional model performance, by implementing Snowflake-schema modelling and governed data structures.
  • Enabled natural-language querying over platform metadata, as measured by query response accuracy, by integrating an LLM API with the Databricks-backed data catalog.
Kubernetes Helm Kustomize ArgoCD Databricks Snowflake
3 months
2025-07 - 2025-09

Medallion Lakehouse Platform

DuckDB MinIO Apache Airflow ...
  • Achieved a 100/100 automated data quality score, as measured by 13 validation tests, by designing a Bronze/Silver/Gold medallion data lake architecture across 5 datasets.
  • Enabled self-service data consumption for technical stakeholders, as measured by API adoption, by building a FastAPI service layer over the curated data.
  • Improved data accessibility for business stakeholders, as measured by dashboard usage, by building a Streamlit dashboard on top of the curated layer.
  • Ensured reliable, auditable data flow from raw to curated layers, as measured by pipeline consistency, by orchestrating ingestion and transformation workflows with Airflow.
DuckDB MinIO Apache Airflow FastAPI Streamlit Pytest

Aus- und Weiterbildung

Aus- und Weiterbildung

2 years 11 months
2022-10 - 2025-08

M.Sc. Data Science (Specialisation in NLP)

GPA: 2.3, Thesis: 1.0, Universität Trier, Germany
GPA: 2.3, Thesis: 1.0
Universität Trier, Germany
3 years 11 months
2017-09 - 2021-07

B.Eng. Computer Science

GPA: 2.0 (German-converted), Visvesvaraya Technological University, India
GPA: 2.0 (German-converted)
Visvesvaraya Technological University, India

Kompetenzen

Kompetenzen

Top-Skills

Python SQL Machine Learning Pyspark AWS Azure Databricks dbt ETL Apache Cassandra Apache Airflow GCP Stakeholdermanagement MS Power BI Kubernetes Terraform GenAI PostgreSQL

Produkte / Standards / Erfahrungen / Methoden

SKILLS

  • Python
  • PySpark
  • Terraform
  • MLflow
  • Time-Series Data Processing
  • MongoDB
  • GitHub Actions & CI/CD
  • Power BI
  • Git
  • SQL
  • Apache Kafka
  • AWS
  • Argo
  • dbt
  • DynamoDB
  • Data Quality & Validation
  • Tableau
  • Cross-Functional Collaboration
  • PostgreSQL
  • Apache Airflow
  • Kubernetes
  • Prometheus & Grafana
  • Docker
  • Data Modelling
  • Medallion Architecture
  • Amazon QuickSight
  • Stakeholder Communication


PROFESSIONAL EXPERIENCE

09/2025 - 03/2026

Role: Business Intelligence Engineer

Customer: Amazon, Germany


Tasks:

  • Kept 500GB+ of analytical data accurate and traceable for 200+ users, as measured by adoption of 10+ governed KPIs across 3 business units, by architecting a governed data lake with data validation and quality control frameworks.
  • Sustained a 99% change-success rate on production pipeline deployments, as measured by release stability, by taking full ownership of monitoring and stabilising production data workflows.
  • Delivered self-service dashboards adopted org-wide, as measured by reduced manual reporting requests, by building governed data models validated against agreed acceptance criteria.
  • Cut manual troubleshooting by 10 hours/week, as measured by reduced incident volume, by setting up proactive data quality monitoring and alerting that flagged deviations before they reached production.
  • Reduced rework on analytical requirements, as measured by validated acceptance criteria, by partnering with business and engineering stakeholders to translate ambiguous asks into reusable data models.


03/2024 - 09/2024

Role: Data Engineer Intern

Customer: Amazon, Luxembourg


Tasks:

  • Cut processing time by 30% for 100+ downstream analysts, as measured by end-to-end pipeline latency across 1TB+ of data, by building near real-time ETL/ELT pipelines integrating 6 enterprise systems into a centralised data lake.
  • Eliminated 95% of manual data-processing intervention, as measured by validation coverage, by implementing automated scheduling, validation, and monitoring across production pipelines.
  • Reduced query latency by 15% across distributed environments, as measured by system performance under load, by optimising large-scale SQL and PySpark transformations over high-volume, timestamped operational datasets.
  • Accelerated onboarding of new data sources, as measured by reusable pipeline components, by documenting system interfaces and data flows for clean handover.


08/2021 - 09/2022

Role: Data Engineer (A.S.E)

Customer: Tata Consultancy Services, India


Tasks:

  • Contributed to a 20% reduction in production defects, as measured by defect recurrence rate, by building modular Python data ingestion and SQL transformation pipelines with automated testing and Git version control.
  • Saved 8 hours/week in manual data preparation across a team of 12, as measured by pipeline throughput, by maintaining 5+ REST API integrations with structured technical documentation.
  • Compressed defect turnaround from 3 days to same-day, as measured by throughput across 50+ Jira-tracked tickets, by investigating and resolving recurring data quality issues.
  • Improved stakeholder decision-making, as measured by adoption, by designing 15+ operational dashboards translating complex system data into actionable insights for non-technical teams.

Einsatzorte

Einsatzorte

Deutschland
möglich

Projekte

Projekte

3 months
2026-01 - 2026-03

Real-Time Retail Data Pipeline

Kafka PySpark Airflow ...
  • Processed 50 events per second of continuous, timestamped streaming data, as measured by throughput across 6 Kafka topics, by architecting a real-time ingestion pipeline with PySpark and Airflow orchestration into a PostgreSQL-backed data lake.
  • Achieved a 100/100 data quality score, as measured by automated validation coverage, by implementing 19 dbt tests across transformation layers handling high-frequency time-stamped records.
  • Cut operational overhead by 40%, as measured by reduced manual workflow effort, by containerising the full pipeline with Docker and a CI/CD deployment process.
  • Reduced end-to-end ingestion-to-availability latency to 38 seconds, as measured by pipeline throughput timing, by tuning PySpark transformation and Airflow scheduling logic for continuous data arrival.
Kafka PySpark Airflow dbt PostgreSQL MinIO Docker CI/CD
5 months
2025-09 - 2026-01

LakeCatalog-AI

Kubernetes Helm Kustomize ...
  • Sustained 99%+ platform uptime, as measured by automated deployment reliability, by architecting a Kubernetes GitOps pipeline (Helm/Kustomize/ArgoCD) hosting a Databricks-backed metadata Q&A agent.
  • Reduced compute costs by 35%, as measured by resource efficiency, by applying domain-oriented Data Mesh design principles.
  • Enabled fast, self-service data querying, as measured by dimensional model performance, by implementing Snowflake-schema modelling and governed data structures.
  • Enabled natural-language querying over platform metadata, as measured by query response accuracy, by integrating an LLM API with the Databricks-backed data catalog.
Kubernetes Helm Kustomize ArgoCD Databricks Snowflake
3 months
2025-07 - 2025-09

Medallion Lakehouse Platform

DuckDB MinIO Apache Airflow ...
  • Achieved a 100/100 automated data quality score, as measured by 13 validation tests, by designing a Bronze/Silver/Gold medallion data lake architecture across 5 datasets.
  • Enabled self-service data consumption for technical stakeholders, as measured by API adoption, by building a FastAPI service layer over the curated data.
  • Improved data accessibility for business stakeholders, as measured by dashboard usage, by building a Streamlit dashboard on top of the curated layer.
  • Ensured reliable, auditable data flow from raw to curated layers, as measured by pipeline consistency, by orchestrating ingestion and transformation workflows with Airflow.
DuckDB MinIO Apache Airflow FastAPI Streamlit Pytest

Aus- und Weiterbildung

Aus- und Weiterbildung

2 years 11 months
2022-10 - 2025-08

M.Sc. Data Science (Specialisation in NLP)

GPA: 2.3, Thesis: 1.0, Universität Trier, Germany
GPA: 2.3, Thesis: 1.0
Universität Trier, Germany
3 years 11 months
2017-09 - 2021-07

B.Eng. Computer Science

GPA: 2.0 (German-converted), Visvesvaraya Technological University, India
GPA: 2.0 (German-converted)
Visvesvaraya Technological University, India

Kompetenzen

Kompetenzen

Top-Skills

Python SQL Machine Learning Pyspark AWS Azure Databricks dbt ETL Apache Cassandra Apache Airflow GCP Stakeholdermanagement MS Power BI Kubernetes Terraform GenAI PostgreSQL

Produkte / Standards / Erfahrungen / Methoden

SKILLS

  • Python
  • PySpark
  • Terraform
  • MLflow
  • Time-Series Data Processing
  • MongoDB
  • GitHub Actions & CI/CD
  • Power BI
  • Git
  • SQL
  • Apache Kafka
  • AWS
  • Argo
  • dbt
  • DynamoDB
  • Data Quality & Validation
  • Tableau
  • Cross-Functional Collaboration
  • PostgreSQL
  • Apache Airflow
  • Kubernetes
  • Prometheus & Grafana
  • Docker
  • Data Modelling
  • Medallion Architecture
  • Amazon QuickSight
  • Stakeholder Communication


PROFESSIONAL EXPERIENCE

09/2025 - 03/2026

Role: Business Intelligence Engineer

Customer: Amazon, Germany


Tasks:

  • Kept 500GB+ of analytical data accurate and traceable for 200+ users, as measured by adoption of 10+ governed KPIs across 3 business units, by architecting a governed data lake with data validation and quality control frameworks.
  • Sustained a 99% change-success rate on production pipeline deployments, as measured by release stability, by taking full ownership of monitoring and stabilising production data workflows.
  • Delivered self-service dashboards adopted org-wide, as measured by reduced manual reporting requests, by building governed data models validated against agreed acceptance criteria.
  • Cut manual troubleshooting by 10 hours/week, as measured by reduced incident volume, by setting up proactive data quality monitoring and alerting that flagged deviations before they reached production.
  • Reduced rework on analytical requirements, as measured by validated acceptance criteria, by partnering with business and engineering stakeholders to translate ambiguous asks into reusable data models.


03/2024 - 09/2024

Role: Data Engineer Intern

Customer: Amazon, Luxembourg


Tasks:

  • Cut processing time by 30% for 100+ downstream analysts, as measured by end-to-end pipeline latency across 1TB+ of data, by building near real-time ETL/ELT pipelines integrating 6 enterprise systems into a centralised data lake.
  • Eliminated 95% of manual data-processing intervention, as measured by validation coverage, by implementing automated scheduling, validation, and monitoring across production pipelines.
  • Reduced query latency by 15% across distributed environments, as measured by system performance under load, by optimising large-scale SQL and PySpark transformations over high-volume, timestamped operational datasets.
  • Accelerated onboarding of new data sources, as measured by reusable pipeline components, by documenting system interfaces and data flows for clean handover.


08/2021 - 09/2022

Role: Data Engineer (A.S.E)

Customer: Tata Consultancy Services, India


Tasks:

  • Contributed to a 20% reduction in production defects, as measured by defect recurrence rate, by building modular Python data ingestion and SQL transformation pipelines with automated testing and Git version control.
  • Saved 8 hours/week in manual data preparation across a team of 12, as measured by pipeline throughput, by maintaining 5+ REST API integrations with structured technical documentation.
  • Compressed defect turnaround from 3 days to same-day, as measured by throughput across 50+ Jira-tracked tickets, by investigating and resolving recurring data quality issues.
  • Improved stakeholder decision-making, as measured by adoption, by designing 15+ operational dashboards translating complex system data into actionable insights for non-technical teams.

Vertrauen Sie auf Randstad

Im Bereich Freelancing
Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

Fragen?

Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

Das Freelancer-Portal

Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.