Working with on AD- Vantage Program, on self-driving car data
Developing Data pipelines using Spark and Airflow for self-driving cars
Generating Metrics for Geospatial applications
Ingesting Data into Elastic search using Apache Spark
Functional Programming with Scala
Automate Azure Kubernetes clusters deployment
Create and deploy Spark Jobs with pytorch + GPUs on Kubernetes
Perform GPU Inferencing on TB?s of data
Working with R&D on data from cars to perform TensorFlow GPU trainings
Developing Data pipelines using Airflow and Apache Spark
Architecture for Migration from Mesos to Kubernetes
Jenkins pipelines for building Docker images to be used Mesos on GPU clusters
Several Infrastructure tasks done on ansible for High Availability
Architect the whole platform
Developing Data pipelines using Airflow and Azure Cloud
Developing the Architecture for the Data Pipelines between on premise and cloud
Writing Spark jobs to clean and aggregate data
In this project we are ingesting huge amounts of data via Kafka Into Accumulo. All the Hadoop environment is Kerberized.
Writing Kafka Connectors to ingest Data
Kerberizing Applications to Hadoop / Kafka / Kafka Connect
Creating statistics plans for RDF4J Query over Accumulo
Creating Apache Nifi Workflows
Introducing git flow Automation, Continuous Integration and Docker Automation
Kafka Connect Setup with Kerberos on Google Kubernetes
Writing Java Applications based on RDF (web semantics)
In this project I had the role Hadoop Architect, some of the tasks were sizing Hadoop Cluster and bringing internal clients to the shared platform and supporting the different Data pipelines flows. All tools were used with a Kerberized Hadoop Cluster
Data Migration using Sqoop and Oozie
Configuring Hadoop Cluster with Kerberos and Active Directory
Implementing Data pipelines using Kylo, Apache Nifi and Talend
Deploying Hortonworks Cloud Break into Amazon AWS
Apache Storm Streaming implementations
Supporting internal clients with streaming and data cleaning operations
Hadoop Sizing for On Premise and on Amazon Cloud
In this project the main goal is to integrate spark deeper into Hbase and Architecting new alerting and computing framework based on Spark Streaming. Every deployment is based on Docker.
Creating Reports in Spark Jobs over history data
Custom Spark Data sources for Hbase and Aggregation for Data exploration
This project consists in designing and implementing Big Data Architecture on Amazon Web services using telecommunications data. This project includes Geospatial operation on Spark written in Scala and a Rest API to Spark Within this project I am also responsible for the following:
Migrating Data from AWS Redshift to Spark which improves speed and decreases cost
Using Hadoop within Amazon Web Services to deploy Spark applications
Writing geospatial applications in Scala on Spark
Working on 3 weekly sprints within an Agile environment
Leading Spark training workshops for colleagues
Responsible for the Service availability from the SAP Systems on the company. We have more then 200 Systems to maintain. Some of the activities that I have done was:
SAP and Oracle Upgrades
SAP OS / HW Migration
Automation scripts for system copies.
TREX Enterprise Search, ASCS Splits, SAP Security, SSO, SNC, SSFS
SAP Fiori with SAP Gateway and SAP Mobile platform.
Responsible for the Service availability from the SAP Systems on the company. We have more then 200 Systems to maintain. Some of the activities that I have done was:
SAP and Oracle Upgrades
SAP OS / HW Migration
Automation scripts for system copies.
TREX Enterprise Search, ASCS Splits, SAP Security, SSO, SNC, SSFS
SAP Fiori with SAP Gateway and SAP Mobile platform.
2012
Master in Networking and Communication
Instituto Politécnico do Porto
Porto, Portugal
2010
Bachelor in Informatics Engineering
Instituto Politécnico do Porto
Porto, Portugal
Training
2020-05
Microsoft Certified: Azure Fundamentals
2019-08
Data Engineering Nanodegree
2016-10
Functional Programming Principles in Scala on Coursera
2016-04
Big Data Analytics Fraunhofer IAIS
2016-02
Databricks Developer Training for Apache Spark
Machine Learning with Big Data by University of California, San Diego on Coursera
Hadoop Platform and Application Framework by University of California on Coursera
Big Data Analytics by University of California, San Diego on Coursera
2012-04
ITL Foundation v4
2012-05
SAP NetWeaver AS Implementation und Operation I (SAP TADM10)
SAP NetWeaver Portal - Implementation and Operation (TEP10)
2013-07
SAP Database Administration I (Oracle) (ADM 505)
SAP Database Administration II (Oracle) (ADM 506)
2014-08
SAP Active Defense Security (AD680)
2013-03
ABAP Performance Tuning (BC 490)
2014-04
SAP Security Days 2014 (WSECUD)
Profile
I do consulting on Cloud solutions Architectures. Over 5 years I have experience with AWS and Azure cloud.
I?m a fan of designing self-service systems to allow people to access data faster, this can only happen with Automation.
The first page of my cv is an overview and not all projects are listed.
For more details information see additional pages.
Software Skills
Scala
Java
Python
Ansible
Kubernetes
Cloud
Linux
Docker
Framework Skills
Apache Spark
Apache Kafka
Apache Nifi
Apache Airflow
Elasticsearch
SAP
SAP Skills:
RFC
SNC
Charm
Kernel Upgrades
EHP Upgrade
SSFS
SSO
HANA
Others:
Puppet
OpenStack
Mesos
SAP Basis
Cloud Technologies:
AWS EMR
AWS S3
AWS Redshift
Google App Engine
Azure Kubernetes
Azure containers
Work Experience
2020-03 - 2020-06
Role: Scala Developer
Customer: BMW AG, München
Tasks:
Creating Geospatial reporting for self-driving cars data. Spark was used to crunch TB?s of data and elasticsearch to index and perform visualization. All components run on OpenShift and apache Airflow.
2018-06 - 2020-02
Role: Enterprise Architect
Customer: Daimler AG, Stuttgart
Tasks:
Lead Architect on DevOps Automation.
Skills:
Jenkins, Kubernetes and CICD
2019-10 - 2019-11
Role: Big Data Architect & Cloud
Customer: s. Oliver GmbH, Würzburg
Tasks:
Design and implement mixed workload on premise and in Azure cloud, based on Containers and Spark jobs to perform Web Crawling.
2017-10 - 2018-06
Role: Spark and Kafka Developer
Customer: Gfk, Nürnberg
Tasks:
Designing data pipelines using Confluent Kafka Connect, Apache Spark and Accumulo. CI / CD was used to do automation and Kubernetes to run the stack. Introduced git flow as standard development flow for teams.
2017-04 - 2017-09
Role: Big Data DevOps
Customer: Deutsche Bahn, Frankfurt
Tasks:
In this project I had the role Big Data Architect, some of the tasks were sizing Hadoop Cluster and bringing internal clients to the shared platform and supporting the different Data pipelines flows. All tools were used with a Kerberized Hadoop.
2016-10 - 2017-03
Role: Java developer
Customer: Kiwigrid, Dresden
Tasks:
Developing custom Spark data sources for HBase. Integrating Spark Jobs on a Vertx Cluster. Designing warehouse for historical data. Migrating data from Mysql to HBase as timeseries.
2016-03 - 2016-09
Role: Big Data Developer
Customer: Here Maps (Ex Nokia), Berlin
Tasks:
Designing and implementing Big Data Architecture on Amazon web services (AWS) using telecommunications data. This project includes Geospatial operation on Spark written in Scala and a Rest API to Spark.
Automotive
Media
Working with on AD- Vantage Program, on self-driving car data
Developing Data pipelines using Spark and Airflow for self-driving cars
Generating Metrics for Geospatial applications
Ingesting Data into Elastic search using Apache Spark
Functional Programming with Scala
Automate Azure Kubernetes clusters deployment
Create and deploy Spark Jobs with pytorch + GPUs on Kubernetes
Perform GPU Inferencing on TB?s of data
Working with R&D on data from cars to perform TensorFlow GPU trainings
Developing Data pipelines using Airflow and Apache Spark
Architecture for Migration from Mesos to Kubernetes
Jenkins pipelines for building Docker images to be used Mesos on GPU clusters
Several Infrastructure tasks done on ansible for High Availability
Architect the whole platform
Developing Data pipelines using Airflow and Azure Cloud
Developing the Architecture for the Data Pipelines between on premise and cloud
Writing Spark jobs to clean and aggregate data
In this project we are ingesting huge amounts of data via Kafka Into Accumulo. All the Hadoop environment is Kerberized.
Writing Kafka Connectors to ingest Data
Kerberizing Applications to Hadoop / Kafka / Kafka Connect
Creating statistics plans for RDF4J Query over Accumulo
Creating Apache Nifi Workflows
Introducing git flow Automation, Continuous Integration and Docker Automation
Kafka Connect Setup with Kerberos on Google Kubernetes
Writing Java Applications based on RDF (web semantics)
In this project I had the role Hadoop Architect, some of the tasks were sizing Hadoop Cluster and bringing internal clients to the shared platform and supporting the different Data pipelines flows. All tools were used with a Kerberized Hadoop Cluster
Data Migration using Sqoop and Oozie
Configuring Hadoop Cluster with Kerberos and Active Directory
Implementing Data pipelines using Kylo, Apache Nifi and Talend
Deploying Hortonworks Cloud Break into Amazon AWS
Apache Storm Streaming implementations
Supporting internal clients with streaming and data cleaning operations
Hadoop Sizing for On Premise and on Amazon Cloud
In this project the main goal is to integrate spark deeper into Hbase and Architecting new alerting and computing framework based on Spark Streaming. Every deployment is based on Docker.
Creating Reports in Spark Jobs over history data
Custom Spark Data sources for Hbase and Aggregation for Data exploration
This project consists in designing and implementing Big Data Architecture on Amazon Web services using telecommunications data. This project includes Geospatial operation on Spark written in Scala and a Rest API to Spark Within this project I am also responsible for the following:
Migrating Data from AWS Redshift to Spark which improves speed and decreases cost
Using Hadoop within Amazon Web Services to deploy Spark applications
Writing geospatial applications in Scala on Spark
Working on 3 weekly sprints within an Agile environment
Leading Spark training workshops for colleagues
Responsible for the Service availability from the SAP Systems on the company. We have more then 200 Systems to maintain. Some of the activities that I have done was:
SAP and Oracle Upgrades
SAP OS / HW Migration
Automation scripts for system copies.
TREX Enterprise Search, ASCS Splits, SAP Security, SSO, SNC, SSFS
SAP Fiori with SAP Gateway and SAP Mobile platform.
Responsible for the Service availability from the SAP Systems on the company. We have more then 200 Systems to maintain. Some of the activities that I have done was:
SAP and Oracle Upgrades
SAP OS / HW Migration
Automation scripts for system copies.
TREX Enterprise Search, ASCS Splits, SAP Security, SSO, SNC, SSFS
SAP Fiori with SAP Gateway and SAP Mobile platform.
2012
Master in Networking and Communication
Instituto Politécnico do Porto
Porto, Portugal
2010
Bachelor in Informatics Engineering
Instituto Politécnico do Porto
Porto, Portugal
Training
2020-05
Microsoft Certified: Azure Fundamentals
2019-08
Data Engineering Nanodegree
2016-10
Functional Programming Principles in Scala on Coursera
2016-04
Big Data Analytics Fraunhofer IAIS
2016-02
Databricks Developer Training for Apache Spark
Machine Learning with Big Data by University of California, San Diego on Coursera
Hadoop Platform and Application Framework by University of California on Coursera
Big Data Analytics by University of California, San Diego on Coursera
2012-04
ITL Foundation v4
2012-05
SAP NetWeaver AS Implementation und Operation I (SAP TADM10)
SAP NetWeaver Portal - Implementation and Operation (TEP10)
2013-07
SAP Database Administration I (Oracle) (ADM 505)
SAP Database Administration II (Oracle) (ADM 506)
2014-08
SAP Active Defense Security (AD680)
2013-03
ABAP Performance Tuning (BC 490)
2014-04
SAP Security Days 2014 (WSECUD)
Profile
I do consulting on Cloud solutions Architectures. Over 5 years I have experience with AWS and Azure cloud.
I?m a fan of designing self-service systems to allow people to access data faster, this can only happen with Automation.
The first page of my cv is an overview and not all projects are listed.
For more details information see additional pages.
Software Skills
Scala
Java
Python
Ansible
Kubernetes
Cloud
Linux
Docker
Framework Skills
Apache Spark
Apache Kafka
Apache Nifi
Apache Airflow
Elasticsearch
SAP
SAP Skills:
RFC
SNC
Charm
Kernel Upgrades
EHP Upgrade
SSFS
SSO
HANA
Others:
Puppet
OpenStack
Mesos
SAP Basis
Cloud Technologies:
AWS EMR
AWS S3
AWS Redshift
Google App Engine
Azure Kubernetes
Azure containers
Work Experience
2020-03 - 2020-06
Role: Scala Developer
Customer: BMW AG, München
Tasks:
Creating Geospatial reporting for self-driving cars data. Spark was used to crunch TB?s of data and elasticsearch to index and perform visualization. All components run on OpenShift and apache Airflow.
2018-06 - 2020-02
Role: Enterprise Architect
Customer: Daimler AG, Stuttgart
Tasks:
Lead Architect on DevOps Automation.
Skills:
Jenkins, Kubernetes and CICD
2019-10 - 2019-11
Role: Big Data Architect & Cloud
Customer: s. Oliver GmbH, Würzburg
Tasks:
Design and implement mixed workload on premise and in Azure cloud, based on Containers and Spark jobs to perform Web Crawling.
2017-10 - 2018-06
Role: Spark and Kafka Developer
Customer: Gfk, Nürnberg
Tasks:
Designing data pipelines using Confluent Kafka Connect, Apache Spark and Accumulo. CI / CD was used to do automation and Kubernetes to run the stack. Introduced git flow as standard development flow for teams.
2017-04 - 2017-09
Role: Big Data DevOps
Customer: Deutsche Bahn, Frankfurt
Tasks:
In this project I had the role Big Data Architect, some of the tasks were sizing Hadoop Cluster and bringing internal clients to the shared platform and supporting the different Data pipelines flows. All tools were used with a Kerberized Hadoop.
2016-10 - 2017-03
Role: Java developer
Customer: Kiwigrid, Dresden
Tasks:
Developing custom Spark data sources for HBase. Integrating Spark Jobs on a Vertx Cluster. Designing warehouse for historical data. Migrating data from Mysql to HBase as timeseries.
2016-03 - 2016-09
Role: Big Data Developer
Customer: Here Maps (Ex Nokia), Berlin
Tasks:
Designing and implementing Big Data Architecture on Amazon web services (AWS) using telecommunications data. This project includes Geospatial operation on Spark written in Scala and a Rest API to Spark.
Automotive
Media
Direktester geht's nicht! Ganz einfach Freelancer finden und direkt Kontakt aufnehmen.