Software Engineer II

Business Group Overview

Do you want to build the systems that deliver generative AI to the world at massive scale? Our team builds and operates the cloud services that run large-scale AI model inference for customers across the globe, processing enormous volumes of requests reliably and efficiently. You will work where distributed systems, capacity, and performance meet, solving problems that only exist at hyperscale.

Job Purpose Overview

As a Software Engineer on our team, you will design and deliver backend services that process high-volume, asynchronous AI workloads and intelligently direct traffic to the right compute at the right time. This role gives you deep, hands-on exposure to distributed systems, large-scale scheduling, capacity and throughput optimization, and service reliability, with room to grow into adjacent parts of the platform. You'll learn to build and operate services that stay fast, efficient, and dependable under intense scale, working alongside a talented and supportive team in a flexible hybrid environment (up to 50% work from home).

Microsoft's mission is to empower every person and every organization on the planet to achieve more, and we're dedicated to this mission across every aspect of our company. Our culture is centered on embracing a growth mindset and encouraging teams and leaders to bring their best each day. Join us and help shape the future of the world.

Responsibilities

- Design, build, and operate scalable, reliable backend services that process high-volume, asynchronous AI inference workloads.

- Develop intelligent scheduling and request-routing capabilities that make efficient, load-aware use of available compute capacity.

- Improve throughput, latency, and cost efficiency of large-scale workloads through profiling, tuning, and thoughtful system design.

- Instrument services with strong telemetry, monitoring, and alerting, and participate in on-call rotations to ensure high availability.

- Partner with product and platform teams to onboard new models and capabilities and to support growing enterprise adoption.

- Write clean, testable, maintainable code and contribute actively to design discussions and code reviews.

- Continuously identify and drive improvements in reliability, scalability, and engineering quality.