Software Engineer, Evaluation

Summary

At Apple, great ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and there’s no telling what you could accomplish. Do you want to make Siri and Apple Intelligence features smarter for our users? Contribute to the next revolution in AI and Machine Learning. We're a dynamic team seeking engineers who are enthusiastic about influencing the future of Apple Intelligence.

Description

As a Software Engineer in AIML Evaluation, you will:

- Work closely with product managers, backend engineers, data scientists and other stakeholders to improve the overall quality and user experience of numerous Apple products.
- Design and develop evaluation system and framework to enhance the quality of Apple Intelligence, ensuring our AI/ML models meet the highest standards.
- Tuning evaluation infrastructure, tightening operational controls, improving auditability and debug trails, and scaling the workflows .
- You’ll play a pivotal role in developing Apple Intelligence, driving the next generation of groundbreaking products across all Apple platforms.

Responsibilities

Benchmark Design & Development: Design and implement evaluation benchmarks, metrics, and test suites that rigorously measure model capabilities across reasoning, knowledge, code, and agentic workflows.
Product-Aligned Evaluation: Develop evaluation methods that capture how models behave in real product settings, and validate that evaluation metrics predict user-perceived quality and product outcomes.
Evaluation Methodology Research & Tooling: Research and apply state-of-the-art evaluation techniques — including scoring frameworks, model-based judging, and contamination-resistant benchmark design. Build reusable tools, scorer libraries, and analysis frameworks that scale across the team’s benchmark portfolio.
Experimental Analysis: Design and execute rigorous experiments comparing model capabilities, engage with third-party vendors on benchmarking, and perform detailed gap analysis to guide model development priorities.
Cross-Team Collaboration: Work closely with model training, training data, and product teams to ensure evaluation insights inform training strategies, data decisions, and product quality improvements.

Minimum Qualifications

3+ years of professional software engineering experience building and operating production systems
Ability to tackle complex challenges, think critically, and deliver innovative solutions.
Fluent use of AI-assisted development tools (coding agents, code review assistants, etc.) to work effectively at scale
Experience designing and scaling distributed systems (load balancing, active-active topologies, capacity planning, throughput-bound services)
Experience with frontend web development (React and modern JavaScript frameworks)
Experience with RESTful API design, SQL and NoSQL databases
BS, MS in Computer Science, Computer Engineering, or equivalent degree with applicable experience

Preferred Qualifications

Eager to explore and apply cutting-edge tech solutions.
Experience working on large, cross-functional, and multi-team projects.
Proven ability to operate independently on ambiguous, open-ended problems where the right answer is not obvious.
Striving to elevate every aspect of the user experience.

Pay & Benefits — Cupertino, California, United States

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $150,400 and $277,600, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

Application Deadline

Apple accepts applications to this posting on an ongoing basis.