Senior Manager Quality Engineering

Category | ---|--- | Partner closely with product, engineering, and data science to embed a "quality-first" mindset in every stage of the development lifecycle for our agentic systems. Act as the key subject matter expert on AI quality, effectively communicating risks, strategies, and results to stakeholders at all levels. Process & Tooling | Evolve our test automation frameworks to handle the non-deterministic and dynamic nature of AI-driven software. Drive the selection and implementation of new tools and infrastructure required for robustly testing and monitoring AI models and agents in production. | Lead, mentor talented QA engineers, fostering a culture of innovation, collaboration, and continuous improvement. Set a clear vision for how quality is defined and achieved in an AI-driven software development environment. AI-Driven Quality Strategy | Develop and implement a comprehensive strategy for developing and testing products built leveraging agentic AI and for using GenAI to enhance testing itself. Pioneer and define a new suite of quality and performance metrics for Generative and Agentic AI models, focusing on areas like accuracy, bias, safety, relevance, and autonomous task completion. Design novel testing methodologies to validate the autonomous behavior of Agentic AI systems within our development process, ensuring their reliability and alignment with intended goals. Bachelor's degree in Computer Science, Engineering, or a related field. 10+ years of experience in Quality Assurance, with at least 5 years in a leadership role. Proven track record of building and leading high-performing QA teams in a fast-paced, agile environment. Strong foundational knowledge of software development lifecycle and modern testing methodologies. Master's degree or PhD in Computer Science, AI, or a related field. Demonstrated expertise in defining and implementing novel quality metrics for AI models, such as fluency, coherence, factuality, hallucination rates, and task success for AI agents. Deep understanding of the concepts behind Agentic AI workflows and the unique challenges associated with testing autonomous, goal-oriented systems. Hands-on experience with AI/ML technologies, particularly Generative AI (e.g., LLMs), and their application to software testing. Experience with a variety of test automation tools and frameworks (e.g., Selenium, Playwright, Cypress) and an understanding of how they need to adapt for AI. Excellent communication, leadership, and interpersonal skills, with the ability to influence and educate a diverse range of stakeholders.