Specialist, QA Test Automation, Northwest Newfoundland/Eastern Labrador (Mary's Harbour)

Position Summary

We are looking for a Specialist, QA Test Automation to bridge the gap between our strategic AI QA vision and day‑to‑day technical execution. Working closely with the QA Chapter Lead, you will embed within Cogeco’s AI squads to bring our AI QA strategy to life, handling probabilistic outputs, dataset curation, and leveraging advanced AI evaluation tools.Key Responsibilities

Probabilistic Testing&Evaluation

Implement and refine testing workflows for AI‑powered solutions where outputs are probabilistic. Build guardrails for regression detection, model drift, prompt regression, hallucination risks, and safety violations. Curate and maintain high‑quality golden datasets that serve as the source of truth for benchmarking model changes.Champion programmatic evaluation tools such as Promptfoo, DeepEval, or Ragas to automate prompt variant and model performance assessment.

Strategy Execution&Collaboration

Translate high‑level AI QA frameworks into actionable tasks for technical teams. Provide real‑time feedback on strategy effectiveness and recommend pivots based on results. Collaborate with DevOps specialists to embed AI evaluation metrics into CI/CD pipelines.

Automation&CI/CD Integration

Design smart pipelines that automatically gate AI responses within modern CI/CD workflows. Validate data pipelines from ingestion to inference, ensuring data integrity.

Essential Qualifications

College diploma or degree in Computer Science, Engineering, or related field (or equivalent experience). 3+ years of experience in Quality Assurance, Software Engineering, or Data Engineering. Proficiency in scripting with Python or JavaScript/TypeScript. Experience with test automation frameworks, API testing, and code repositories (Git).Familiarity with CI/CD tools such as Bitbucket pipelines, Google Cloud Run, GitHub Actions, GitLab CI, or Jenkins. Desired Competencies

Analytical Deep‑Dives: Diagnose unexpected AI behavior through data and prompt analysis. Tool Adaptability: Quickly learn and implement new AI evaluation and observability tooling. Growth Mindset: Thrive in ambiguous, high‑uncertainty environments and execute roadmaps independently. Collaboration&Communication: Clearly convey testing gaps to AI developers and product owners. Benefits

Flexibility with remote and on‑site options. Competitive compensation package. Comprehensive benefits including health, vision, dental, and retirement plans. Discounts on company services for employees and families. Location

Burlington, ON Company

Cogeco Connexion Inc. EEO Statement

Cogeco Inc. is committed to diversity and inclusion. We encourage individuals of all backgrounds to apply. If you need any accommodations during the application process, please contact us confidentially at .

#J-18808-Ljbffr