Senior Data Scientist - Outlook Science Team

The Outlook team at Microsoft is reimagining how people communicate, organize their work, and manage time through intelligent, trustworthy experiences across Mail and Calendar. We build products used at global scale, including Copilot-powered experiences for drafting, summarization, search, inbox prioritization, meeting preparation, scheduling, and agentic workflows. Our work spans product analytics, experimentation, telemetry, machine learning, generative AI, and large language model evaluation to improve customer value, quality, trust, and adoption across Outlook experiences. 

The Outlook Data Science team is looking for a Senior Data Scientist to lead high-visibility initiatives across Outlook and Copilot. In this role, you will partner with product, engineering, design, research, and applied science teams to define success, build trusted measurement systems, evaluate AI experiences, and translate complex data into product decisions. You will be expected to operate as a strategic thought partner, provide technical leadership, and help establish rigorous data science practices across the organization. 

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

• Lead the design and execution of analytical frameworks that evaluate product investments across Outlook Mail, Calendar, Search, and Copilot experiences. 
• Define metrics, value-creating interactions, guardrails, funnels, retention measures, and feature-level success criteria in partnership with product managers, engineers, and scientists. 
• Design and analyze online experiments, quasi-experiments, and observational studies to estimate causal impact and guide ship decisions. 
• Develop trusted dashboards, semantic models, and reporting pipelines that monitor product health, experimentation outcomes, AI quality, reliability, latency, sentiment, and adoption. 
• Evaluate generative AI and agentic experiences using offline and online evaluation methods, including benchmark datasets, human judgment, LLM-as-a-judge approaches, error analysis, and production telemetry. 
• Apply prompt engineering and evaluation techniques to improve response quality, grounding, relevance, safety, and task completion for Outlook scenarios. 
• Partner with applied scientists and engineers on model selection, feature development, post-training signals, and measurement strategies for AI-powered product experiences.
• Use large-scale behavioral and telemetry data to identify customer needs, diagnose product issues, forecast trends, and uncover opportunities across commercial and consumer segments. 
• Influence product strategy by communicating clear, actionable insights to senior leaders and translating ambiguous business questions into measurable analytical plans. 
• Provide technical mentorship and raise the bar for statistical rigor, reproducibility, privacy-conscious analytics, and operational excellence across the data science community. 

• Experience with Generative AI technologies, such as large language models (LLMs), retrieval-augmented generation (RAG), agentic systems, and natural language processing (NLP), with an understanding of AI quality dimensions such as relevance, groundedness, factuality, safety, reliability, latency, and user trust.
• Hands-on experience in AI evaluation and optimization, such as prompt engineering, benchmark and rubric design, LLM evaluation, human-in-the-loop assessment, A/B testing, offline evaluations, production telemetry analysis, and leveraging customer feedback to improve AI product performance.
• Experience working with large, distributed datasets and building reusable analytical datasets, metrics layers, dashboards, or self-service analytics solutions. 
• Advanced proficiency with big-data technologies and cloud analytics platforms. Experience with Azure Data Explorer/Kusto, Azure Data Lake, Synapse, Databricks, Data Factory, or Power BI is a plus. 
• Experience driving product analytics for communication, productivity, search, collaboration, or enterprise software products, with a proven ability to lead ambiguous cross-functional initiatives, influence stakeholders, communicate insights effectively, and deliver measurable business impact at scale.
• Advanced degree in a quantitative or technical field is preferred.
• 6+ years of experience in data science, statistics, machine learning, product analytics, business intelligence, or data-driven product strategy.
• 4+ years of experience using SQL and Python or R to conduct analysis and implement statistical or machine learning methods, such as regression, classification, clustering, forecasting, causal inference, or experimentation.
• Experience defining product metrics and using large-scale data to influence roadmap, investment, and ship decisions.
• Experience designing experiments and applying statistical methods to measure product or business impact.