Designed and implemented a fully instrumented, cloud-native observability and telemetry framework for hosted, fine-tuned, and proxied LLMs operating in enterprise-grade production environments. Delivered end-to-end visibility into AI/ML training pipelines, inference workloads, and model serving infrastructure. Developed unified tracing, logging, and metrics pipelines leveraging distributed-systems observability standards to surface granular insights across token-level usage, GPU/CPU resource saturation, container/node performance, latency distributions (P50?P99), error propagation, and emergent model-behavior ?flares.? Integrated cross-cloud observability stacks to support proactive SRE practices, automated incident triage, drift detection, and optimization of high-throughput AI workloads.

• Architected end-to-end observability pipelines for LLM APIs and model-serving runtimes using OpenTelemetry SDKs/collectors, Prometheus exporters, and Kubernetes operators, instrumenting the full lifecycle of model training, inference, and system-level resource utilization.
• Instrumented model endpoints, batch/stream training jobs, and inference gateways to capture high-resolution metrics such as tail latency, throughput (RPS/QPS), token-per-second performance, GPU memory fragmentation, multi-node utilization, error budgets, and anomaly detection signals via statistical and ML-based detectors.
• Implemented automated alerting and SLO/SLA monitoring using Prometheus Alertmanager, and custom anomaly-detection pipelines to identify inference latency regressions, GPU/CPU saturation events, memory leaks, container restarts, or failed model-training runs?reducing mean time to detect (MTTD) and mean time to resolve (MTTR).
• Conducted architectural reviews, telemetry schema standardization, and iterative refinements to reduce observability overhead, optimize scrape intervals, improve sampling strategies, and enhance distributed tracing efficiency across multi-cloud LLM deployments.
• Collaborated with ML Ops, SRE, and platform engineering teams to integrate telemetry into CI/CD pipelines, automate environment drift detection, and enable data-driven scaling policies for training and inference clusters.

• Built service broker for Cloud Foundry using Node.js.
• Developed billing and metering microservices to track and charge resource usage.
• Implemented Prometheus/Grafana monitoring to ensure system reliability.
• Designed APIs for seamless microservice integration and external communication.
• Ensured service scalability, supporting hundreds of concurrent tenants reliably.

• Developed Terraform and Ansible playbooks for installation and automated upgrades.
• Built a lightweight agent to respond to Consul changes, triggering relevant playbooks.
• Implemented APIs for customer HANA system orders.
• Designed secure VPC architecture and automated backup/recovery workflows using AWS S3 and Glacier.
• Reduced deployment time by 40% while maintaining compliance and security standards.