Sr. Machine Learning Engineer, Speech LLM Evaluation

In this role, you help shape speech LLM evaluation for Apple's AI assistants, focusing on accuracy, robustness, and conversational quality. You will build evaluation datasets and metrics that reflect real usage, collaborating with modeling, infrastructure, and product teams to ensure rigorous, scalable assessment before release. Your work supports trustworthy, private-by-design interactions across Apple devices. This is a chance to impact how users interact with Siri and Apple Intelligence in a privacy-centric context.

• Comprehensive medical and dental coverage
• Retirement benefits
• Employee stock programs
• Tuition reimbursement
• Relocation assistance
• Discounted products and free services

• Own data and metrics foundation for evaluating speech LLMs across accuracy, robustness, and conversational quality
• Build and curate evaluation datasets reflecting real usage (personalized queries, multi-turn conversations)
• Design metrics and automated judges translating model outputs into actionable signals
• Collaborate with modeling, infrastructure, and product teams to ensure rapid, consistent, rigorous evaluation before customers

• Bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience
• Experience building or working with text, speech or audio evaluation pipelines and metrics
• Proficiency in Python and building data processing pipelines at scale
• Experience curating or annotating datasets for ML evaluation or training
• Working knowledge of statistics for evaluating model performance
• Familiarity with large language model evaluation techniques, including automated (LLM-as-judge) and human evaluation methods
• Strong written and verbal communication skills to explain evaluation results to technical and non-technical audiences

• strong written and verbal communication
• ability to explain evaluation results to diverse audiences
• cross-functional collaboration
• Python
• data processing pipelines at scale
• dataset curation/annotation for ML evaluation