Designed and implemented an observability and telemetry framework for production LLM workloads (hosted and proxied) in enterprise environments. Focused on backend architecture, metrics pipelines, and standardized telemetry to give engineering and SRE teams insight into performance, reliability, and cost of AI services across multi-cloud deployments.

• Designed the overall observability architecture for LLM gateways and supporting microservices running on Kubernetes, with a clear separation of concerns for metrics, logging, and tracing.
• Instrumented services end?to?end using OpenTelemetry to capture traces, latency, token usage, and error profiles for LLM requests and downstream dependencies.
• Built metrics pipelines backed by Prometheus and integrated dashboards/alerts in Grafana and Promitor to support SLO/SLA tracking, anomaly detection, and incident response.
• Defined telemetry schemas and APIs for internal consumers (platform, product, SRE teams), enabling consistent querying and aggregation across services and environments.
• Worked with SRE and platform teams to tune sampling, retention, and alert thresholds to balance observability depth with system performance and operational cost.

Developed backend microservices to offer SAP Analytics Cloud as a SaaS product via the Cloud Foundry marketplace. Work focused on service broker APIs, metering, and billing services that integrate with downstream capacity management and system provisioning systems.

• Designed and implemented the Cloud Foundry service broker, providing lifecycle APIs (provision, bind, update, deprovision) for SAC instances.
• Developed metering and billing microservices in Go, exposing REST APIs to capture usage, calculate charges, and integrate with internal billing and capacity systems.
• Introduced an API gateway to standardize authentication, routing, and rate limiting across the broker, metering, and billing endpoints.
• Used Redis and Postgres to persist service instances, usage records, and billing data with appropriate indexing, constraints, and migration strategies.
• Instrumented services with Prometheus metrics and logging to monitor availability, latency, and throughput, and to support troubleshooting and capacity planning.
• Collaborated with product and finance stakeholders to align technical metering models with commercial pricing structures.

• Developed Terraform and Ansible playbooks for installation and automated upgrades.
• Built a lightweight agent to respond to Consul changes, triggering relevant playbooks.
• Implemented APIs for customer HANA system orders.
• Designed secure VPC architecture and automated backup/recovery workflows using AWS S3 and Glacier.
• Reduced deployment time by 40% while maintaining compliance and security standards.

• Designed and implemented the caching service in , exposing a clear API for cache reads/writes and integrating with upstream analytics components.
• Modeled cache keys and data structures to account for user roles, permissions, and cube dimensions, ensuring correct and secure cache hits.
• Implemented cache invalidation strategies (e.g., key-based and pattern-based invalidation) to handle data changes without stale results.
• Used Redis as the primary cache store and MongoDB for underlying metadata/state where needed.
• Deployed the service on Kubernetes with appropriate resource limits, autoscaling policies, and rolling upgrade strategies.
• Instrumented the service with Prometheus metrics (hit rate, latency, error rates, resource usage) to optimize performance and capacity.