Staff AI Test Automation Engineer

Hi, we’re Gravie. Our mission is to create health benefits that actually benefit small and midsize businesses and their employees. Our innovative benefit solutions and services are developed and delivered by a diverse group of unique people. We encourage you to be your authentic self - we like you that way. We improve the way employers offer health benefits and provide employees coverage they can use. The Staff AI Test Automation Engineer at Gravie will take hands-on ownership of our AI-driven test automation systems across our product portals. The Playwright framework and migration from legacy tooling is already underway, we need someone who can pick up this work, drive it to completion, and continuously improve, scale, and optimize it as we go. Rather than manually writing test cases, you will build on the frameworks, feedback loops, and infrastructure that enable AI tools to generate, execute, and maintain automated tests at scale. You will work closely with our QA, Engineering, and Product teams to dramatically increase our automated regression coverage, accelerate delivery timelines, and reduce the risk of defects reaching production. Agentic AI is central to this role — you'll be hands-on with AI agents that generate, execute, and iterate on tests with minimal manual intervention. CORE RESPONSIBILITIES: - Continue building and hands-on improve AI-driven test automation workflows using tools like Claude Code, where AI agents generate, execute, and iterate on Playwright tests with minimal human intervention. Your primary job is building and refining the system, not writing individual tests. - Continue evolving the existing Playwright test framework, refining patterns, page-object models, fixtures, and conventions to make them more efficient, scalable, and better optimized for AI-assisted test generation. - Build and improve tight feedback loops between local test execution, CI pipeline results, and merge request creation, so AI agents can run tests, interpret failures, fix issues, and resubmit with minimal manual intervention. - Drive the ongoing migration from Geb/Spock to Playwright to completion, using AI agents to accelerate conversion at scale while maintaining coverage continuity. - Integrate and continuously optimize Playwright test execution in GitLab CI/CD pipelines with automated quality gates on every merge request and deployment. Optimize for fast feedback through parallelization and caching strategies. - Instrument tests with reporting on pass/fail trends, flakiness rates, and coverage gaps. Use this data to prioritize AI-driven test generation toward the highest-value uncovered areas. - Identify scenarios where end-to-end Playwright tests are not the right tool and build API-layer automation or contract tests for microservice boundaries instead. - Build hands-on automation for claims systems, contract testing, system integrations, and end-to-end user flow journeys. - Continuously improve the AI automation workflow by refining prompts, context structures, and validation patterns to increase the percentage of tests that AI generates correctly on the first pass. QUALIFICATIONS: - 10+ years of experience in test automation engineering, SDET, or software engineering with strong testing practices. - Hands-on experience with Agentic AI tools (Claude Code, Cursor, GitHub Copilot, or similar) to generate and maintain code at scale. This is not optional; we need someone who has already built workflows around AI-assisted development. - Hands-on experience with Playwright or a comparable modern end-to-end testing framework. - Proficiency in TypeScript or JavaScript with the ability to write production-quality code. - Experience with CI/CD pipelines (GitLab preferred), including writing pipeline configuration, test stages, and merge request workflows. - Solid understanding of REST API testing and when to use API-level tests vs. UI-level tests. - Hands-on experience building test automation for claims systems, contract testing, integrations, and user flow journeys. - Experience migrating legacy test automation to a modern framework, including maintaining coverage during the migration. - Strong communication skills: able to explain technical trade-offs to non-technical stakeholders and write clear documentation. - A builder mindset. You think in terms of systems, feedback loops, and automation leverage, not individual test cases. - Comfort working in ambiguity. Our AI-driven approach is new, and the right person will help define what good looks like. NICE TO HAVE: - Experience with Groovy, Spock, or Geb (the legacy framework you will be migrating from). - Experience with contract testing (Pact or equivalent). - Background in health insurance, benefits administration, or regulated industries. - Experience building or contributing to internal developer tooling or productivity platforms. A LITTLE MORE ABOUT US: - We know healthcare. Our company was founded and is still led by industry veterans who have started and grown several market-leading companies in the space. - We have raised money from top tier investors who share the same long-term vision as we do of building an industry defining company that will endure over the long run. We are well capitalized. - Our clients love us. Customer satisfaction rates among employees using Gravie health plans consistently rank above 80% – nearly 40 points above the industry average. - Our culture is unique. We tend to be non-hierarchical, merit-driven, opinionated but kind people who thrive working in a high-performance, fast-paced environment. People at Gravie care deeply about making a positive impact in the lives of the people we serve. BENEFITS Our unique benefits program is the gravy, i.e., the special sauce that sets our compensation package apart. In addition to standard health and wellness benefits, Gravie’s package includes alternative medicine coverage, flexible PTO, up to 16 weeks paid parental leave, paid holidays, a 401k program, transportation perks, education reimbursement, and 2 days of paid paw-ternity leave. JOB APPLICANTS If you apply for employment with Gravie, personal information collected via our applicant tracking vendor is subject to our standalone California Job Applicant Notice at Collection, accessible directly within the application workflow and separate from this Policy.