AI Quality Engineer

Global Financial Firm located in Irving, TX has an immediate contract opportunity for an experienced AI Quality Engineer

"This role is currently on a Hybrid Schedule.
You will need to have reliable internet, computer and android or iphone for remote access into the client systems during remote work.
We will be expected in the office weekly 3 days depending on the team requirement.

****Video/ f2f interviews are required prior to all offers.

Pay rate range: $ 74.00 - $ 78.00 Negotiable based upon years of experience

Job Description:
We are seeking a highly skilled and detail-oriented AI Quality Engineer to design, develop, and implement comprehensive quality assurance strategies for our artificial intelligence applications and AI agents. The ideal candidate will be responsible for creating robust test plans and procedures to validate the performance, functionality, accuracy, and resilience of AI models and systems. You will play a critical role in ensuring our AI solutions are reliable, robust, and deliver intended value in production environments.
Responsibilities:

• Quality Assurance Strategy & Standards: Design and develop overall quality assurance plans, procedures, and standards specifically tailored for AI/Machine Learning models and systems, including Generative AI solutions and autonomous AI agents.
• Test Strategy & Plan Development: Create detailed test strategies and test plans to thoroughly evaluate the robustness, functionality, performance, and ethical behavior of AI agents and models. This includes defining test objectives, scope, resources, and schedules.
• Test Case Design & Execution: Develop, implement, and execute a wide range of test cases (functional, non-functional, performance, stress, regression, adversarial testing) to validate AI model outputs, decision-making processes, and overall system behavior.
• AI Robustness & Functionality Testing: Focus on verifying AI agent robustness against various inputs, edge cases, and unexpected scenarios, ensuring they consistently deliver accurate and reliable results while adhering to design specifications.
• Performance & Scalability Testing: Design and conduct tests to measure AI model inference speed, resource utilization, and scalability under different load conditions.
• Data Quality & Model Bias Assessment: Contribute to ensuring data quality for training and validation, and work with data scientists to design tests for identifying and mitigating model bias and fairness issues.
• Automation Development: Develop and maintain automated testing frameworks and scripts for AI/ML pipelines, model validation, and system integration.
• Defect Management: Identify, document, track, and prioritize defects and inconsistencies in AI model behavior and system functionality, working closely with development teams for timely resolution.
• Metric Definition & Reporting: Define key quality metrics (e.g., accuracy, precision, recall, F1-score, latency, reliability) and provide regular status updates and quality reports to stakeholders.
• Collaboration & Communication: Collaborate effectively with data scientists, machine learning engineers, software developers, product managers, and business stakeholders to understand requirements and integrate quality throughout the AI development lifecycle.
• Continuous Improvement: Stay abreast of the latest advancements in AI testing methodologies, tools, and best practices, and actively contribute to the continuous improvement of our AI development and deployment processes.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Software Engineering, Quality Assurance, or a related technical field.
• 3+ years of experience in Quality Assurance or Software Testing, with at least 1-2 years specifically focused on AI/Machine Learning systems.
• Strong understanding of AI/ML concepts, including supervised, unsupervised, and reinforcement learning, as well as knowledge of different AI model types (e.g., Generative AI, predictive models, classification models).
• Experience in designing and developing test strategies, test plans, and test cases for complex software systems, particularly those involving AI components.
• Proficiency in at least one programming language (Python strongly preferred) for test automation and data analysis.
• Familiarity with AI/ML frameworks (e.g., TensorFlow, PyTorch) and relevant testing libraries.
• Experience with various testing methodologies and tools (e.g., unit testing, integration testing, API testing, performance testing tools).
• Excellent analytical skills and attention to detail, with the ability to identify subtle anomalies in AI behavior.
• Strong communication and collaboration skills, with the ability to articulate technical issues clearly.