Get the Hyriko app Instant job alerts and faster browsing — install on your phone.
© 2026 Hyriko. Listings sourced from public job boards.
Associate Director | Hybrid cloud | Bengaluru | Engineering | Hybrid Cloud Engineering at Deloitte South Asia · Hyriko
via Career pages · 3w ago
Associate Director | Hybrid cloud | Bengaluru | Engineering | Hybrid Cloud Engineering Deloitte South Asia
Full-time On-site
Location: Bengaluru, India Type: Full-time Posted: 3w ago
Role Overview We are seeking a seasoned Site Reliability Engineering (SRE) and Observability leader to design, build, and scale reliability frameworks for AI/GenAI platforms and data-intensive workloads.
This role will focus on ensuring high availability, performance, scalability, and cost-efficiency across AI infrastructure (LLMs, model training/inference, vector databases, pipelines) by embedding SRE principles, observability, and automation into the platform lifecycle.
Key Responsibilities SRE Strategy for AI Infrastructure Define and lead SRE strategy and operating model for AI platforms across cloud (Azure, AWS, GCP) and hybrid environments Establish SLIs, SLOs, and SLAs tailored to: LLM inference latency and throughput Model training performance and job success rates Pipeline reliability (RAG, orchestration frameworks, agents) Drive adoption of error budgets and reliability engineering practices across AI and platform teams Observability Architecture for AI Workloads Design and implement end-to-end observability frameworks for AI systems, including: Metrics (latency, throughput, GPU utilization, token usage) Logs (model behavior, system failures, prompt traces)
Don't want to miss the next one?
Subscribe to daily email alerts for roles matching your interests.
Get email alerts
Traces (distributed AI workflows, API calls, orchestration flows)
- Build observability for: LLM pipelines and agent-based systems Vector databases and retrieval layers Data ingestion and feature pipelines Enable deep visibility into model performance, drift, and degradation Reliability Engineering & Automation Implement self-healing systems, auto-remediation, and resiliency patterns
- Design fault tolerance strategies: Model fallback and routing strategies Graceful degradation in GenAI systems
- Lead adoption of: Chaos engineering for AI workloads Canary deployments and A/B testing for models Drive automation-first SRE practices using IaC and policy-as-code AI System Performance Optimization
- Optimize: Inference latency and throughput GPU/accelerator utilization Distributed training efficiency
- Work with engineering teams to: Fine-tune model serving infrastructure Implement caching, batching, and async processing Drive performance benchmarking frameworks for AI workloads Incident Management & Reliability Operations Establish incident response frameworks tailored for AI platforms Lead root cause analysis (RCA) for: Infrastructure bottlenecks Define and track MTTR, MTBF, availability, and reliability KPIs Build runbooks, playbooks, and operational dashboards Tooling & Platform Enablement Implement and manage observability and SRE tooling such as: Monitoring: Prometheus, Grafana, Datadog, Azure Monitor, CloudWatch Logging & tracing: ELK stack, OpenTelemetry, Jaeger AI observability: Langfuse, Weights & Biases, Arize, WhyLabs (preferred) Develop custom telemetry pipelines for AI-specific metrics (token usage, prompt traces, response quality signals) Integrate observability into CI/CD and MLOps pipelines Governance & Risk Management Define reliability guardrails and governance policies Ensure compliance, security, and availability requirements for AI systems
- Implement controls for: Model drift and degradation detection Responsible AI monitoring Stakeholder Leadership & Advisory Act as a trusted advisor to: Enterprise architecture and leadership Translate reliability metrics into business impact (customer experience, revenue risk) Drive enterprise adoption of SRE practices for AI Thought Leadership & Innovation Develop POVs, frameworks, and accelerators for: Observability patterns for GenAI
- Stay ahead of trends in: AI reliability engineering Observability tooling and standards Lead internal capability building and external client workshops
- 12–15+ years of experience in: Site Reliability Engineering / DevOps / Platform Engineering Cloud infrastructure and distributed systems 4–6+ years working with AI/ML platforms, MLOps, or data-intensive systems Proven experience in designing high-scale, highly reliable systems
- Deep expertise in: SRE principles (SLI/SLO, error budgets, incident management) Observability (metrics, logs, tracing) Distributed system design and failure modes
- Strong understanding of: AI/ML workloads (training, inference, pipelines) LLM architectures and GenAI systems Cloud Platforms: Azure, AWS, GCP
- Infrastructure: Kubernetes, containers, serverless architectures
- Observability stack: OpenTelemetry, Prometheus, Grafana, ELK
- Programming / scripting:
- CI/CD & IaC: Terraform, ARM, CloudFormation, GitOps Leadership & Consulting Skills
Executive communication and stakeholder management Ability to lead cross-functional, global teams Strong problem-solving and analytical mindset Experience in client-facing advisory and transformation programs
- Certifications: Cloud Architect (Azure/AWS/GCP)
- Exposure to: AI observability platforms (Arize, WhyLabs, Langfuse, etc.) FinOps alignment for AI workloads
- Experience with: Multi-cloud and hybrid deployment strategies
Similar roles you might like More openings like this one — take a look before you go.
Browse all jobs 