Description: Development of data pipelines to backfill historical data and stream daily incoming data

Responsibilities:

? Development of the backend services, review code quality

? Testing

? Deployment

? Reporting

Tech stack: Kubernetes, Apache Iceberg, AWS, Terraform, Spark, Scala, Python, RisingWave 

Description:

• Development of an AI multi-agents application (backend in Python - frontend in Angular - infrastructure hosted on kubernetes)
• Several services developed in the backend: CRUD application to manage user chats history with

Responsibilities:

? Development of the backend services

? CRUD service to manage user chats history with MongoDB

? development of several AI services: summarization and document comparison

? development of a Python internal shared library

? Description: Development of an agentic LLM Python application for retail customer advice

? Responsibilities:

? Onboarding customer data sets into Vector stores

? Context augmentation using Langchain framework

? Classification, extraction, summarization and document matching

? Workshops on architecture with LLM and RAG implementation

? Tech stack: Python, Azure, LangChain, PostgreSQL 

• Benchmarking of Generative AI application development
  
• Development of a webapp RAG application using Streamlit for the frontend
• Context augmentaion using Langchain framework
• Classification, extraction, summarization and document matching
• Use of google cloud services: VertexAI, gcs, DocumentAI, cloudfront
• Workshop trainer on LLM and RAG topic 

• Develop new features in Front and Back end
  
• Develop unit and integration tests using TDD methodology
• Review other people?s code
• Teach other people best coding practices

• Data Cleaning and feature engineering on customer data with the client developer team

• Spark and Scala training of the customer data team
• Implementation of a Spark preprocessing data pipeline connected to AWS for external data sources
• Error Analysis on critical ETL Data flows
• Improving Data Pipelines and SQL Setup
• Smaller Development Tasks
• Understanding customer needs and analysis of the business needs

• Preparation and delivery of a Spark and Scala workshop
• Spark data processing
• Introduction to SparkMlLib 

•  Data Cleaning and feature engineering on customer data
• Migration of MongoDB backed up data to parquet files in AWS bucket
• Implementation of a Spark Machine learning application to automate feature Engineering from streamed data (SQS) and build machine learning prediction models, written in Scala
• Implementation of a scala API that makes predictions from our trained and loaded machine learning models
• Batch shell scripts cronjobs to automate data migration to PostgreSQL
• Error Analysis on critical ETL Data flows
• Performing Maintenance Tasks
• User Communication
• Improving Data Pipelines and SQL Setup
• Smaller Development Task
• Customer Orientation and Communication: Strong communication and willingness to investigate Issues start to finish to provide a satisfactory user experience
• Analytical and Problem-Solving Orientation: Demonstrable ability to tackle problems with a logical, analytical, and data-driven approach. Keen to develop complex problem-solving skills

• Develop Frontend application using Scala.js
• Use of Outwatch as a reactive Web-Frontend Library for Scala.js
• Terraform scripting for the deployment
• AWS deployment

• Backend data pipeline to stream data from legacy sytem to PostgreSQL using Debezium
• GrapQL interfacing to query data from the frontend
• Development of a web and mobile application in Flutter to interact with customer products

• Backend data pipeline to stream media between partners
• Implement APIs and SDKs
• Implementation of multiple backend microservices

• Backend implementation in collaboration with the data scientists team
• Aggregation of multiple external data sources
• Implement APIs and SDKs
• Implementation of multiple backend microservices