QA Engineer (Senior) ID35731

AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software solutions that help companies across 15+ industries change the lives of millions.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment every day, there’s no better place - guaranteed! :)


WHAT YOU WILL DO

- Architect and oversee end-to-end test strategies across different layers in the testing pyramid;
- Define, evaluate, and monitor model validation criteria, including accuracy threshold, confidence scoring, fallback behaviors, etc.;
- Drive the automation of QA processes, ensuring robust, scalable processes for data ingestion, extraction, and classification;
- Lead the validation integration points between AI components and business logic, identifying risks and ensuring seamless operation;
- Own, maintain, and evolve the test infrastructure, specifically the CI/CD quality gates, data generation, and automated validation scripts;
- Establish and monitor key quality metrics aligned with the business goals, including turnaround time, confidence scores, and corrections;
- Act as a strategic partner to Product and Engineering, and advocate for quality from design through delivery;
- Keep up with the latest AI tech and how to best apply the latest advancements to make the previously impossible possible for our customers;
- Sense of ownership, take ownership of code quality through exploratory and automated tests.

MUST HAVES

- Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field experience;
- 5+ years in AQA, Software Test Engineering, or related roles, including leadership responsibilities;
- Experience in AQA Javascript;
- Solid knowledge of software development life cycle (SDLC), Agile methodologies, and Quality Assurance or Quality Engineering methodologies;
- Proven experience building and scaling test automation frameworks (e.g. Pytest, Cypress, Selenium, Postman, etc.). Strong technical skills with REST APIs, data, pipelines, and backend services;
- Strong technical background with APIs, CI/CD pipelines, backend services, and cloud environments (AWS, Azure);
- Exceptional attention to detail, with a mindset for building robust systems that catch issues early and protect the customer experience;
- Demonstrated fast, proactive learning skills, especially crucial in the rapidly changing field of AI;
- Uppper-Intermediate English level.

NICE TO HAVES

- Experience testing AI/ML in production environments;
- Experience with GenAI model evaluation metrics;
- Experience setting up proactive monitoring and alerting systems to catch issues like model drift, performance degradation, or workflow failures.

THE BENEFITS OF ING US

- Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roaps.

- Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.

- A selection of exciting projects: projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.

- Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.


Your application doesn't end here! To unlock the next steps, check your email and complete your registration on our Applicant Site. The incomplete registration results in the termination of your process.