Senior Software Engineer

At Microsoft, we’re a community of passionate innovators driven by curiosity and purpose. We collaborate to imagine what’s possible and accelerate our careers in a cloud-powered world where openness and innovation unlock limitless potential. 

Artificial Intelligence is central to Microsoft’s strategy—and the Azure AI Platform is leading the charge. As part of our team, you’ll contribute to cutting-edge projects that solve real-world challenges using transformative technologies. 

We are looking for a Senior Software Engineer to join our fast-paced, agile team at the core of Microsoft’s AI infrastructure. This team builds and operates the Next Generation Traffic Shaping, Scheduling, and Optimization Platform—a foundational control-plane service responsible for intelligent routing, capacity allocation, quota enforcement, and workload optimization for OpenAI models and other large-scale AI workloads across Azure. 

In this role, you will help define and build the systems that determine how AI traffic is routed and served globally. You will work on the Traffic Shaper, the critical platform that dynamically balances demand across thousands of AI endpoints, optimizes utilization of premium AI accelerators, and ensures reliable, low-latency experiences for Microsoft Copilot, Azure OpenAI Service, and strategic enterprise customers. 

You will design and develop large-scale distributed systems responsible for: 

• Intelligent request routing and workload placement across global regions. 

• Capacity management and allocation across models, tenants, and customer offerings. 

• Real-time traffic shaping, prioritization, and load balancing. 

• Quota management, throttling, and fairness enforcement. 

• Fleet-wide optimization to maximize GPU utilization and service efficiency. 

• High-fidelity telemetry, observability, and automated operational controls. 

• Resiliency, failover, and incident mitigation mechanisms for mission-critical AI services. 

As a senior engineer, you will provide technical leadership across projects, drive architecture decisions, influence platform strategy, and partner closely with teams across Azure AI, OpenAI, Infrastructure, Networking, and Core AI Services. You will own services operating at global scale and help shape the future of Microsoft's AI infrastructure. 

• Design and implement highly scalable, reliable, and performant distributed systems that power Azure AI traffic management and scheduling. 

• Drive architecture and technical direction for routing, capacity allocation, and workload optimization services. 

• Develop algorithms and control-plane systems that intelligently distribute AI workloads across global infrastructure while meeting latency, availability, and cost objectives. 

• Collaborate with partner teams to onboard new models, regions, and customer offerings while maintaining operational excellence. 

• Build observability and operational tooling to monitor service health, capacity utilization, traffic patterns, and customer impact in near real-time. 

• Lead investigations into complex production issues and drive systematic improvements to reliability, resiliency, and platform automation. 

• Identify and execute opportunities to improve fleet efficiency, reduce operational overhead, and optimize utilization of premium AI infrastructure. 

• Mentor engineers, review designs and code, and raise engineering standards across the organization. 

• Drive engineering excellence through testing, automation, documentation, and operational best practices. 

• Influence long-term platform strategy and contribute to roadmap planning for next-generation AI infrastructure. 

Required: 

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

   

Preferred: 

• Experience operating large-scale services with stringent reliability and availability requirements. 

• Experience with distributed storage systems, telemetry platforms, and real-time data processing. 

• Familiarity with AI/ML infrastructure, GPU fleets, inference serving systems, or large-scale model deployment platforms. 

• Experience leading cross-team initiatives and driving projects from design through production deployment and operation. 

#AIPLATFORM, #AzureOpenAI, #llm, #distributedsystems, #MicrosoftAI, #AICoreServices  

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.