Automation Testing with Gen AI , RAG , LLM

We are looking for a Quality Assurance Engineer who prides themselves on a functional-first mindset. You are someone who thinks deeply about user journeys, business logic, and "what happens if I click this?" before even writing a script. In this role, you will be responsible for the end-to-end quality of our healthcare automation platformensuring that the core infrastructure is rock-solid while pioneering ways to test our Generative AI integrations.

Key Responsibilities

1. Core Functional Excellence

• End-to-End Testing: Own the functional validation of complex healthcare workflows (e.g., RCM, Patient Access, and Clinical Coding).
• Test Strategy: Design comprehensive test plans and cases that cover happy paths, edge cases, and negative scenarios based on deep business logic.
• API & Data Validation: Perform rigorous API testing and verify data integrity across our platforms, ensuring that backend services communicate flawlessly.
• Regression & Stability: Maintain a robust regression suite to ensure that new AI deployments don't break existing, critical healthcare features.

2. GenAI & LLM Evaluation

• Non-Deterministic Testing: Move beyond "Pass/Fail" to evaluate the quality, relevance, and safety of LLM-generated responses.
• RAG Validation: Test the accuracy of our Retrieval-Augmented Generation (RAG) systemsensuring the AI is pulling the correct clinical data and citing it accurately.
• Failure Analysis: Investigate why the AI might fail in downstream processing (e.g., OCR extraction errors or logic hallucinations) and work with devs to refine prompts.
• Model Comparison: Help benchmark different models (OpenAI, Gemini, Llama) to ensure functional consistency as we swap or upgrade engines.

What You Bring

• The "Detective" Mindset: You don't just find bugs; you find the root cause. You have a proven ability to think through complex functional logic.
• Core Toolkit: 35 years of experience in QA with proficiency in SQL, API testing (Postman/RestAssured), and Automation (Selenium/Playwright).