AI & Data Engineer mit Schwerpunkt Databricks (PySpark, Delta Lake) und Generative AI (RAG, Agentic AI, Multi-Agent-Systeme) im Microsoft-Azure-Umfeld
Aktualisiert am 26.06.2026
Profil
Freiberufler / Selbstständiger
Remote-Arbeit
Verfügbar ab: 01.07.2026
Verfügbar zu: 100%
davon vor Ort: 100%
Databricks
Azure
Generative AI
PySpark
Delta Lake
Unity Catalog
Python
SQL
Agentic AI
LangChain
MCP
FastAPI
Azure Data Factory
Azure Data Lake Storage
Azure Entra ID
Terraform
MLflow
Power BI
Data Engineering
Machine Learning
Git
BigQuery
RAG
German
C2 ? Native
English
C1 ? Fluent

Einsatzorte

Einsatzorte

Deutschland, Schweiz, Österreich
möglich

Projekte

Projekte

3 months
2026-04 - 2026-06

Lakehouse Infrastructure as Code & Secret Management

Data Engineer
Data Engineer

Stabilized and evolved the Terraform-managed Lakehouse infrastructure across multiple environments in a regulated enterprise setting. Resolved a systemic state-versus-runtime drift in credential and secret management spanning Terraform state, Key Vault, and Databricks identity, restoring deterministic, idempotent deployments. Governed staged Azure DevOps pipelines with controlled plan/apply separation and approval gates, surfacing and de-risking production-critical changes ahead of rollout.

Terraform Azure DevOps Pipelines Azure Key Vault Databricks Unity Catalog Azure Data Factory ADLS Gen2 Entra ID Git
MHK
Berlin
2 months
2026-03 - 2026-04

CWS API Integration ? Creditreform WebServices

Data Engineer Databricks Delta Lake Unity Catalog ...
Data Engineer
Implemented a cloud-based ingestion pipeline for the Creditreform CWS SOAP API: Python client library, Azure Functions for mailbox processing, ADF orchestration, Databricks notebook for XML parsing and Delta Table upsert into Unity Catalog; full acceptance testing with 17 documented test cases.
Databricks Delta Lake Unity Catalog Azure Data Factory Azure Functions Python SQL
MHK / Healthcare / Insurance
Berlin
2 months
2026-03 - 2026-04

AI-Center: Databricks Agentic AI Integration

AI Engineer Databricks Mosaic AI Genie Spaces ...
AI Engineer
Designed and deployed a Databricks Supervisor Agent (Mosaic AI) orchestrating multiple Genie Spaces and tools through a single governed endpoint; integrated into the corporate AI platform (LibreChat) via MCP and OAuth 2.0 (Azure Entra ID), incl. architecture design, auth troubleshooting, and technical documentation.
Databricks Mosaic AI Genie Spaces MCP OAuth 2.0 Azure Entra ID Python
MHK / Healthcare / Insurance
Berlin
5 months
2025-03 - 2025-07

Automated Nostro Account Reconciliation

Data Engineer / Data Analyst Databricks Spark Python ...
Data Engineer / Data Analyst
Design and implementation of a fully automated reconciliation process matching internal ledger bookings with external Nostro account statements at transaction level: reconciliation logic on value dates, debit/credit indicators and tolerances; mismatch categorization; match/mismatch reporting; close collaboration with finance and operations teams.
Azure Data Factory Delta Lake ADLS Gen2
Databricks Spark Python SQL
LBBW / Banking
Berlin
3 months
2024-12 - 2025-02

Scalable Power BI Architecture with Databricks Integration

Data Engineer / Data Analyst Databricks Power BI SQL
Data Engineer / Data Analyst
Conceptual design of a secure, scalable Power BI reporting architecture with Databricks integration: data models and governance concepts, Dev/Test/Prod deployment strategy, RBAC and Row-Level Security, performance optimization, alignment with business and IT stakeholders.
Databricks Power BI SQL
CBTW / Banking
Berlin
7 months
2024-05 - 2024-11

Migration SAS Marketing Automation to SAS CI 360

Data Engineer / Data Analyst SAS CI 360 SAS Base SQL
Data Engineer / Data Analyst
Migration and modernization of marketing-automation processes to the cloud-based SAS CI 360 platform: redesign of data models for cloud compatibility, management of technical environments, coordination between marketing and IT.
SAS CI 360 SAS Base SQL
ERGO / Insurance
Berlin
5 months
2024-02 - 2024-06

ESG Strategy and Dashboard

Data Engineer / Data Analyst Databricks Power BI SQL
Data Engineer / Data Analyst
Development of an enterprise-wide ESG strategy and a central ESG dashboard for management reporting: KPI definition, Power BI dashboards, reporting automation, data preparation and processing on Databricks, CSRD compliance.
Azure Data Factory
Databricks Power BI SQL
CBTW / Consulting / IT
Berlin
5 months
2024-01 - 2024-05

SAS-Based ETL and Data Warehouse Development

Data Engineer / Data Analyst SAS DI SAS Base SQL ...
Data Engineer / Data Analyst

Design and implementation of ETL processes and data-warehouse solutions for large-scale operational data: ETL pipeline development, administration and optimization of DWH structures, data quality and consistency, operational support.

SAS DI SAS Base SQL HTML JavaScript
TotalEnergies / Energy
Berlin

Aus- und Weiterbildung

Aus- und Weiterbildung

2 months
2024-10 - 2024-11

Databricks Fundamentals Accreditation

Databricks
Databricks
2 months
2024-10 - 2024-11

Databricks Certified Data Analyst Associate

Databricks
Databricks
Databricks
2 months
2024-01 - 2024-02

SAS Certified Specialist: Base Programming

SAS
SAS
4 years 1 month
2015-10 - 2019-10

B.Sc. Business Mathematics

Technische Universität Berlin
Technische Universität Berlin
  • Focus: Statistics & Stochastics

Kompetenzen

Kompetenzen

Top-Skills

Databricks Azure Generative AI PySpark Delta Lake Unity Catalog Python SQL Agentic AI LangChain MCP FastAPI Azure Data Factory Azure Data Lake Storage Azure Entra ID Terraform MLflow Power BI Data Engineering Machine Learning Git BigQuery RAG

Produkte / Standards / Erfahrungen / Methoden

PROFILE

Data & AI Engineer with ~3 years of hands-on Databricks experience, combining two complementary strengths: building and migrating data-warehouse workloads to the Databricks Lakehouse ? robust ingestion and processing pipelines in PySpark, Delta Lake, and SQL ? and delivering production GenAI on Databricks: end-to-end RAG pipelines (chunking, embeddings, vector search, re-ranking) and agentic multi-agent architectures (Mosaic AI, Genie Spaces) exposed via OpenAI-compatible APIs. Financial-domain background across banking, insurance, and macroeconomic forecasting for institutional investors. Delivered in regulated enterprise environments with Git-based workflows, Jira-driven change management, IaC (Terraform), and formal client acceptance.


EXPERIENCE

10+ years data & analytics · ~3 years Databricks


TECHNICAL SKILLS

Expert

  • Databricks (PySpark / Delta Lake / Unity Catalog) 
  • Python 
  • SQL 
  • RAG 
  • Agentic AI


Advanced

  • Azure 
  • GCP / BigQuery 
  • Terraform 
  • FastAPI 
  • LangChain 
  • MLflow 
  • Docker 
  • Git 
  • Jira 
  • Power BI


Familiar

  • Tableau 
  • SAS 
  • Excel 
  • Java


PROFESSIONAL EXPERIENCE

10/2022 ? today

Role: Data Engineer / Data Analystn, Freelance


Tasks:

  • Databricks Projects: ETL prototyping and analytical workloads with PySpark and SQL for SME clients ? ongoing in parallel to consulting engagements.
  • AI Automation: LLM-based workflow automation and agentic decision-support tooling for SME and enterprise clients.
  • Statistical Consulting: Advanced methods (regression, clustering, hypothesis testing) for clients in healthcare, finance, and academia.


Skills:

Databricks / Python / SQL / R / FastAPI / Docker / Power BI


02/2026 ? 06/2026

Role: AI Engineer / Data Engineer (Consultant)

Customer: Ultra Tendency


Tasks:

  • Databricks Lakehouse Engineering: Design and development of data-ingestion and processing pipelines with PySpark, Delta Lake, and Unity Catalog; resolved a production Delta-merge deduplication issue and optimized pipeline performance.
  • Source Integration & Scheduling: Integrated external data sources (REST/SOAP APIs, incl. credit-rating data) into the Lakehouse; built reporting and scheduling solutions with Databricks Workflows.
  • Data Engineering for RAG & GenAI: Built the ingestion and curation layer ? sourcing, parsing, and structuring data into Delta / Unity Catalog tables ? feeding GenAI agents (Genie Spaces); the same Delta-based pattern populates RAG vector indexes via Databricks Vector Search; working knowledge of chunking, embeddings, and vector search.
  • Agentic AI on Databricks: Multi-agent architectures (Mosaic AI Agent Framework, Genie Spaces) orchestrated through a single governed endpoint, exposed via OpenAI-compatible REST APIs (FastAPI).
  • Software Engineering & Change Management: Git-based development with code reviews, infrastructure as code (Terraform on Azure), CI/CD, and Jira-based ticket workflows in a regulated enterprise environment.
  • Testing & Acceptance: Unit and integration testing, root-cause analysis, and formal client acceptance incl. documentation of 17 acceptance test cases.


    Skills:

    Databricks / PySpark / Delta Lake / Unity Catalog / Python / SQL / Azure / Terraform / FastAPI / Git / Jira


    01/2024 ? 07/2025

    Role: Data Engineer / Data Analyst (Consultant)

    Customer: CBTW


    Tasks:

    • DWH-to-Lakehouse Migration: Supported the migration of legacy data-warehouse workloads to Databricks; re-implemented ETL logic as Databricks notebooks and jobs (PySpark, SQL).
    • Pipeline Engineering: Developed and optimized ingestion and transformation pipelines over multi-source datasets; improved pipeline efficiency and data quality together with data-engineering teams.
    • Forecasting & Analytics: Built and tuned predictive models; delivered interactive Power BI / Tableau dashboards for financial-services clients in banking and insurance.
    • Delivery Process: Worked in Git/Jira-driven delivery with code reviews, unit tests, and ticket-based change management.


    Skills:

    Databricks / PySpark / Python / SQL / Power BI / Tableau / Git / Jira / SAS


      05/2020 ? 10/2022

      Role: Data Engineer / Data Analyst

      Customer: FocusEconomics


      Tasks:

      • Investment-Grade Market Intelligence: Data-driven macroeconomic reports and forecasts for international financial clients ? banks, asset managers, and institutional investors.
      • Forecasting Models: Classification and regression models predicting macroeconomic developments; visual reporting and systematic code-quality management.


      Skills:

      Python / SQL / Power BI / Tableau


        08/2017 ? 11/2017

        Role: Data Analyst Intern, Controlling

        Customer: tecis Finanzdienstleistungen AG


        Tasks:

        • Financial Modeling & Controlling: Integrated financial models and performance statistics in a regulated financial-services environment.

        Einsatzorte

        Einsatzorte

        Deutschland, Schweiz, Österreich
        möglich

        Projekte

        Projekte

        3 months
        2026-04 - 2026-06

        Lakehouse Infrastructure as Code & Secret Management

        Data Engineer
        Data Engineer

        Stabilized and evolved the Terraform-managed Lakehouse infrastructure across multiple environments in a regulated enterprise setting. Resolved a systemic state-versus-runtime drift in credential and secret management spanning Terraform state, Key Vault, and Databricks identity, restoring deterministic, idempotent deployments. Governed staged Azure DevOps pipelines with controlled plan/apply separation and approval gates, surfacing and de-risking production-critical changes ahead of rollout.

        Terraform Azure DevOps Pipelines Azure Key Vault Databricks Unity Catalog Azure Data Factory ADLS Gen2 Entra ID Git
        MHK
        Berlin
        2 months
        2026-03 - 2026-04

        CWS API Integration ? Creditreform WebServices

        Data Engineer Databricks Delta Lake Unity Catalog ...
        Data Engineer
        Implemented a cloud-based ingestion pipeline for the Creditreform CWS SOAP API: Python client library, Azure Functions for mailbox processing, ADF orchestration, Databricks notebook for XML parsing and Delta Table upsert into Unity Catalog; full acceptance testing with 17 documented test cases.
        Databricks Delta Lake Unity Catalog Azure Data Factory Azure Functions Python SQL
        MHK / Healthcare / Insurance
        Berlin
        2 months
        2026-03 - 2026-04

        AI-Center: Databricks Agentic AI Integration

        AI Engineer Databricks Mosaic AI Genie Spaces ...
        AI Engineer
        Designed and deployed a Databricks Supervisor Agent (Mosaic AI) orchestrating multiple Genie Spaces and tools through a single governed endpoint; integrated into the corporate AI platform (LibreChat) via MCP and OAuth 2.0 (Azure Entra ID), incl. architecture design, auth troubleshooting, and technical documentation.
        Databricks Mosaic AI Genie Spaces MCP OAuth 2.0 Azure Entra ID Python
        MHK / Healthcare / Insurance
        Berlin
        5 months
        2025-03 - 2025-07

        Automated Nostro Account Reconciliation

        Data Engineer / Data Analyst Databricks Spark Python ...
        Data Engineer / Data Analyst
        Design and implementation of a fully automated reconciliation process matching internal ledger bookings with external Nostro account statements at transaction level: reconciliation logic on value dates, debit/credit indicators and tolerances; mismatch categorization; match/mismatch reporting; close collaboration with finance and operations teams.
        Azure Data Factory Delta Lake ADLS Gen2
        Databricks Spark Python SQL
        LBBW / Banking
        Berlin
        3 months
        2024-12 - 2025-02

        Scalable Power BI Architecture with Databricks Integration

        Data Engineer / Data Analyst Databricks Power BI SQL
        Data Engineer / Data Analyst
        Conceptual design of a secure, scalable Power BI reporting architecture with Databricks integration: data models and governance concepts, Dev/Test/Prod deployment strategy, RBAC and Row-Level Security, performance optimization, alignment with business and IT stakeholders.
        Databricks Power BI SQL
        CBTW / Banking
        Berlin
        7 months
        2024-05 - 2024-11

        Migration SAS Marketing Automation to SAS CI 360

        Data Engineer / Data Analyst SAS CI 360 SAS Base SQL
        Data Engineer / Data Analyst
        Migration and modernization of marketing-automation processes to the cloud-based SAS CI 360 platform: redesign of data models for cloud compatibility, management of technical environments, coordination between marketing and IT.
        SAS CI 360 SAS Base SQL
        ERGO / Insurance
        Berlin
        5 months
        2024-02 - 2024-06

        ESG Strategy and Dashboard

        Data Engineer / Data Analyst Databricks Power BI SQL
        Data Engineer / Data Analyst
        Development of an enterprise-wide ESG strategy and a central ESG dashboard for management reporting: KPI definition, Power BI dashboards, reporting automation, data preparation and processing on Databricks, CSRD compliance.
        Azure Data Factory
        Databricks Power BI SQL
        CBTW / Consulting / IT
        Berlin
        5 months
        2024-01 - 2024-05

        SAS-Based ETL and Data Warehouse Development

        Data Engineer / Data Analyst SAS DI SAS Base SQL ...
        Data Engineer / Data Analyst

        Design and implementation of ETL processes and data-warehouse solutions for large-scale operational data: ETL pipeline development, administration and optimization of DWH structures, data quality and consistency, operational support.

        SAS DI SAS Base SQL HTML JavaScript
        TotalEnergies / Energy
        Berlin

        Aus- und Weiterbildung

        Aus- und Weiterbildung

        2 months
        2024-10 - 2024-11

        Databricks Fundamentals Accreditation

        Databricks
        Databricks
        2 months
        2024-10 - 2024-11

        Databricks Certified Data Analyst Associate

        Databricks
        Databricks
        Databricks
        2 months
        2024-01 - 2024-02

        SAS Certified Specialist: Base Programming

        SAS
        SAS
        4 years 1 month
        2015-10 - 2019-10

        B.Sc. Business Mathematics

        Technische Universität Berlin
        Technische Universität Berlin
        • Focus: Statistics & Stochastics

        Kompetenzen

        Kompetenzen

        Top-Skills

        Databricks Azure Generative AI PySpark Delta Lake Unity Catalog Python SQL Agentic AI LangChain MCP FastAPI Azure Data Factory Azure Data Lake Storage Azure Entra ID Terraform MLflow Power BI Data Engineering Machine Learning Git BigQuery RAG

        Produkte / Standards / Erfahrungen / Methoden

        PROFILE

        Data & AI Engineer with ~3 years of hands-on Databricks experience, combining two complementary strengths: building and migrating data-warehouse workloads to the Databricks Lakehouse ? robust ingestion and processing pipelines in PySpark, Delta Lake, and SQL ? and delivering production GenAI on Databricks: end-to-end RAG pipelines (chunking, embeddings, vector search, re-ranking) and agentic multi-agent architectures (Mosaic AI, Genie Spaces) exposed via OpenAI-compatible APIs. Financial-domain background across banking, insurance, and macroeconomic forecasting for institutional investors. Delivered in regulated enterprise environments with Git-based workflows, Jira-driven change management, IaC (Terraform), and formal client acceptance.


        EXPERIENCE

        10+ years data & analytics · ~3 years Databricks


        TECHNICAL SKILLS

        Expert

        • Databricks (PySpark / Delta Lake / Unity Catalog) 
        • Python 
        • SQL 
        • RAG 
        • Agentic AI


        Advanced

        • Azure 
        • GCP / BigQuery 
        • Terraform 
        • FastAPI 
        • LangChain 
        • MLflow 
        • Docker 
        • Git 
        • Jira 
        • Power BI


        Familiar

        • Tableau 
        • SAS 
        • Excel 
        • Java


        PROFESSIONAL EXPERIENCE

        10/2022 ? today

        Role: Data Engineer / Data Analystn, Freelance


        Tasks:

        • Databricks Projects: ETL prototyping and analytical workloads with PySpark and SQL for SME clients ? ongoing in parallel to consulting engagements.
        • AI Automation: LLM-based workflow automation and agentic decision-support tooling for SME and enterprise clients.
        • Statistical Consulting: Advanced methods (regression, clustering, hypothesis testing) for clients in healthcare, finance, and academia.


        Skills:

        Databricks / Python / SQL / R / FastAPI / Docker / Power BI


        02/2026 ? 06/2026

        Role: AI Engineer / Data Engineer (Consultant)

        Customer: Ultra Tendency


        Tasks:

        • Databricks Lakehouse Engineering: Design and development of data-ingestion and processing pipelines with PySpark, Delta Lake, and Unity Catalog; resolved a production Delta-merge deduplication issue and optimized pipeline performance.
        • Source Integration & Scheduling: Integrated external data sources (REST/SOAP APIs, incl. credit-rating data) into the Lakehouse; built reporting and scheduling solutions with Databricks Workflows.
        • Data Engineering for RAG & GenAI: Built the ingestion and curation layer ? sourcing, parsing, and structuring data into Delta / Unity Catalog tables ? feeding GenAI agents (Genie Spaces); the same Delta-based pattern populates RAG vector indexes via Databricks Vector Search; working knowledge of chunking, embeddings, and vector search.
        • Agentic AI on Databricks: Multi-agent architectures (Mosaic AI Agent Framework, Genie Spaces) orchestrated through a single governed endpoint, exposed via OpenAI-compatible REST APIs (FastAPI).
        • Software Engineering & Change Management: Git-based development with code reviews, infrastructure as code (Terraform on Azure), CI/CD, and Jira-based ticket workflows in a regulated enterprise environment.
        • Testing & Acceptance: Unit and integration testing, root-cause analysis, and formal client acceptance incl. documentation of 17 acceptance test cases.


          Skills:

          Databricks / PySpark / Delta Lake / Unity Catalog / Python / SQL / Azure / Terraform / FastAPI / Git / Jira


          01/2024 ? 07/2025

          Role: Data Engineer / Data Analyst (Consultant)

          Customer: CBTW


          Tasks:

          • DWH-to-Lakehouse Migration: Supported the migration of legacy data-warehouse workloads to Databricks; re-implemented ETL logic as Databricks notebooks and jobs (PySpark, SQL).
          • Pipeline Engineering: Developed and optimized ingestion and transformation pipelines over multi-source datasets; improved pipeline efficiency and data quality together with data-engineering teams.
          • Forecasting & Analytics: Built and tuned predictive models; delivered interactive Power BI / Tableau dashboards for financial-services clients in banking and insurance.
          • Delivery Process: Worked in Git/Jira-driven delivery with code reviews, unit tests, and ticket-based change management.


          Skills:

          Databricks / PySpark / Python / SQL / Power BI / Tableau / Git / Jira / SAS


            05/2020 ? 10/2022

            Role: Data Engineer / Data Analyst

            Customer: FocusEconomics


            Tasks:

            • Investment-Grade Market Intelligence: Data-driven macroeconomic reports and forecasts for international financial clients ? banks, asset managers, and institutional investors.
            • Forecasting Models: Classification and regression models predicting macroeconomic developments; visual reporting and systematic code-quality management.


            Skills:

            Python / SQL / Power BI / Tableau


              08/2017 ? 11/2017

              Role: Data Analyst Intern, Controlling

              Customer: tecis Finanzdienstleistungen AG


              Tasks:

              • Financial Modeling & Controlling: Integrated financial models and performance statistics in a regulated financial-services environment.

              Vertrauen Sie auf Randstad

              Im Bereich Freelancing
              Im Bereich Arbeitnehmerüberlassung / Personalvermittlung

              Fragen?

              Rufen Sie uns an +49 89 500316-300 oder schreiben Sie uns:

              Das Freelancer-Portal

              Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.