Designed and implemented a fully instrumented, cloud-native observability and telemetry

framework for hosted, fine-tuned, and proxied AI/ML models in enterprise-grade production

environments. Delivered end-to-end visibility into AI/ML training pipelines, inference workloads,

and model serving infrastructure.

Responsibilities:

?? Architected end-to-end observability pipelines for ML APIs and model-serving runtimes using

OpenTelemetry SDKs/collectors, Prometheus exporters, and Kubernetes operators,

instrumenting the full lifecycle of model training, inference, and system-level resource

utilization.

?? Instrumented model endpoints, batch/stream training jobs, and inference gateways to capture

high-resolution metrics such as tail latency, throughput (RPS/QPS), token-per-second

performance, GPU memory fragmentation, multi-node utilization, error budgets, and anomaly

detection signals.

?? Profiled and monitored inference optimization for vLLM and TensorRT-LLM deployments,

tracking CUDA kernel performance, NCCL communication overhead, and memory bandwidth

utilization to identify and resolve latency regressions.

?? Monitored distributed multi-GPU training runs across nodes connected via InfiniBand,

capturing per-GPU utilization, gradient synchronization bottlenecks, and HPC cluster health

signals for large-scale model training workloads.

?? Implemented automated alerting and SLO/SLA monitoring using Prometheus Alertmanager

and custom anomaly-detection pipelines to identify inference latency regressions, GPU/CPU

saturation events, memory leaks, container restarts, or failed model-training runs.

?? Collaborated with MLOps, SRE, and platform engineering teams to integrate telemetry into

CI/CD pipelines, automate environment drift detection, and enable data-driven scaling policies

for training and inference clusters.

Built and operated a cloud-native, event-driven control-plane platform inspired by Kubernetes, enabling automated provisioning and lifecycle management of distributed workloads across thousands of tenants.

Strong focus on Kubernetes ecosystem tooling, GitOps workflows, observability, and secure, production-grade infrastructure, with hands-on experience designing scalable platforms using infrastructure-as-code and container-native patterns.

Responsibilities:

• Designed and operated Kubernetes-style control-plane components (API server, reconciliation loops, RBAC, resource isolation), aligning closely with operator-based architectures and extensibility patterns.
• Managed containerized workloads across Kubernetes and Docker, applying Helm-based deployments and declarative configuration practices.
• Implemented Git-driven deployment workflows and CI/CD pipelines (GitLab CI/CD, trunk-based development), enabling automated build, test, and release cycles.
• Built event-driven infrastructure using message queues, Redis, and WebSockets to support real-time orchestration and high-throughput distributed systems.
• Automated infrastructure provisioning using AWS CDK and configuration management concepts aligned with tools like Ansible.
• Integrated observability stack with Prometheus, Grafana, and alerting pipelines for metrics, logging, and anomaly detection.
• Implemented persistent storage and state management strategies using SQL databases (PostgreSQL-compatible patterns) with strong consistency and concurrency controls.
• Developed and maintained internal platform tooling and SDKs to standardize deployment workflows and reduce operational complexity for engineering teams.
• Utilized local cloud emulation via LocalStack to improve CI reliability and developer feedback loops.