Highlights
Founded in 1998 and headquartered in Farmington Hills, MI, Kyyba has a global presence delivering high-quality resources and top-notch recruiting services, enabling businesses to effectively respond to organizational changes and technological advances. Evaluate chatbot performance across dimensions defined by ***, including intent, task completion, correctness, completeness, relevance, groundedness, source attribution, response quality, safety, policy compliance, escalation need, and failure category.
Numbers & Facts
| Location | Dearborn, MI |
| Industry | Staffing/Employment Agencies |
| Company Size | 100 to 499 employees |
| Year Founded | 1998 |
| Website | http://www.kyyba.com/ |
Description
Job Title: (Validation Engineer Engineer 2)
About Kyyba:
Founded in 1998 and headquartered in Farmington Hills, MI, Kyyba has a global presence delivering high-quality resources and top-notch recruiting services, enabling businesses to effectively respond to organizational changes and technological advances.
At Kyyba, the overall well-being of our employees and their families is important to us. We are proud of our work culture which embodies our core values; incorporating value, passion, excellence, empowerment, and happiness, creates a vibrant and productive atmosphere. We empower our employees with the resources, incentives, and flexibility that they need to support a healthy, balanced, and fulfilling career by providing many valuable benefits and a balanced compensation structure combined with career development.
Job Description
*Data Annotator* Improve the quality of the chatbot responses by creating a feedback loop where AI responses are judged by humans. Review chatbot conversations and related artifacts including individual turns, model responses, retrieved content, citations, and metadata within ***-approved tools. Apply ***-approved labeling guidelines, taxonomies, and decision rules, completing all required labels, rationales, confidence assessments, and ambiguity indicators. Evaluate chatbot performance across dimensions defined by ***, including intent, task completion, correctness, completeness, relevance, groundedness, source attribution, response quality, safety, policy compliance, escalation need, and failure category. Identify and escalate content that is ambiguous, sensitive, safety-relevant, outside the current guidelines, or requires subject-matter-expert review. Labelers must not infer or create standards where approved guidance does not exist. Participate in onboarding, calibration, refresher training, quality reviews, and adjudication sessions to maintain consistent labeling decisions. Conduct first-level quality checks on assigned work and correct errors identified through *** or supplier quality reviews. Maintain complete and accurate records of completed annotations, quality results, exceptions, escalations, and open issues in ***-designated systems. Provide weekly reports summarizing completed volume, productivity, quality results, backlog, open questions, risks, guideline gaps, and required actions. Recommend and document improvements to the annotation guidelines and taxonomy based on recurring ambiguities, labeling patterns, and adjudication outcomes. Changes may be implemented only after *** approval.
Skills Required:
Analytical, Communications, Data Annotation - Voice and Text
Skills Preferred:
N/A
Experience Required:
Experience labeling conversational AI data, including text and audio. Prior experience in the automotive industry is preferred. Professional fluency in English, with strong reading comprehension and written communication skills. Demonstrated ability to interpret and consistently apply detailed guidelines, taxonomies, and decision rules. Strong analytical judgment and attention to detail, including the ability to identify factual errors, incomplete responses, unsupported claims, policy concerns, and ambiguous cases. Ability to document labeling decisions and provide clear, concise rationales. Ability to work accurately on repetitive tasks while meeting quality, productivity, and turnaround expectations. Ability to participate in calibration sessions, incorporate feedback, and adjust decisions as guidelines evolve. Ability to prepare clear weekly reports summarizing labeling volume, productivity, quality results, progress, and open issues.
Experience Preferred:
Amazon, Google or Apple or Scale AI
Education Required:
Bachelor's Degree
Education Preferred:
Masters
:
Location: (Dearborn,MI)
Disclaimer:
Kyyba is an Equal Opportunity Employer.
Kyyba does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. Minorities / Females / Protected Veterans / Individuals with Disabilities are encouraged to apply. All employment is decided on the basis of qualifications, merit, and business need.”
It is the policy of Kyyba to provide reasonable accommodation when requested by a qualified applicant or employee with a disability, unless such accommodation would cause an undue hardship. The policy regarding requests for reasonable accommodation applies to all aspects of employment, including the application process. If reasonable accommodation is needed, please contact Kyyba at 248-813-9665
Rewards:
Medical, dental, vision
401k
Term life
Voluntary life and disability insurance
Optional Pre-paid legal plan
Optional Identity theft plan
Optional Medical and dependent FSA
Work-visa sponsorship
Opportunity for advancement
Long-term assignment with opportunity for hire by client
SELECT AWARDS
About Kyyba:
Founded in 1998 and headquartered in Farmington Hills, MI, Kyyba has a global presence delivering high-quality resources and top-notch recruiting services, enabling businesses to effectively respond to organizational changes and technological advances.
At Kyyba, the overall well-being of our employees and their families is important to us. We are proud of our work culture which embodies our core values; incorporating value, passion, excellence, empowerment, and happiness, creates a vibrant and productive atmosphere. We empower our employees with the resources, incentives, and flexibility that they need to support a healthy, balanced, and fulfilling career by providing many valuable benefits and a balanced compensation structure combined with career development.
Job Description
*Data Annotator* Improve the quality of the chatbot responses by creating a feedback loop where AI responses are judged by humans. Review chatbot conversations and related artifacts including individual turns, model responses, retrieved content, citations, and metadata within ***-approved tools. Apply ***-approved labeling guidelines, taxonomies, and decision rules, completing all required labels, rationales, confidence assessments, and ambiguity indicators. Evaluate chatbot performance across dimensions defined by ***, including intent, task completion, correctness, completeness, relevance, groundedness, source attribution, response quality, safety, policy compliance, escalation need, and failure category. Identify and escalate content that is ambiguous, sensitive, safety-relevant, outside the current guidelines, or requires subject-matter-expert review. Labelers must not infer or create standards where approved guidance does not exist. Participate in onboarding, calibration, refresher training, quality reviews, and adjudication sessions to maintain consistent labeling decisions. Conduct first-level quality checks on assigned work and correct errors identified through *** or supplier quality reviews. Maintain complete and accurate records of completed annotations, quality results, exceptions, escalations, and open issues in ***-designated systems. Provide weekly reports summarizing completed volume, productivity, quality results, backlog, open questions, risks, guideline gaps, and required actions. Recommend and document improvements to the annotation guidelines and taxonomy based on recurring ambiguities, labeling patterns, and adjudication outcomes. Changes may be implemented only after *** approval.
Skills Required:
Analytical, Communications, Data Annotation - Voice and Text
Skills Preferred:
N/A
Experience Required:
Experience labeling conversational AI data, including text and audio. Prior experience in the automotive industry is preferred. Professional fluency in English, with strong reading comprehension and written communication skills. Demonstrated ability to interpret and consistently apply detailed guidelines, taxonomies, and decision rules. Strong analytical judgment and attention to detail, including the ability to identify factual errors, incomplete responses, unsupported claims, policy concerns, and ambiguous cases. Ability to document labeling decisions and provide clear, concise rationales. Ability to work accurately on repetitive tasks while meeting quality, productivity, and turnaround expectations. Ability to participate in calibration sessions, incorporate feedback, and adjust decisions as guidelines evolve. Ability to prepare clear weekly reports summarizing labeling volume, productivity, quality results, progress, and open issues.
Experience Preferred:
Amazon, Google or Apple or Scale AI
Education Required:
Bachelor's Degree
Education Preferred:
Masters
:
Location: (Dearborn,MI)
Disclaimer:
Kyyba is an Equal Opportunity Employer.
Kyyba does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. Minorities / Females / Protected Veterans / Individuals with Disabilities are encouraged to apply. All employment is decided on the basis of qualifications, merit, and business need.”
It is the policy of Kyyba to provide reasonable accommodation when requested by a qualified applicant or employee with a disability, unless such accommodation would cause an undue hardship. The policy regarding requests for reasonable accommodation applies to all aspects of employment, including the application process. If reasonable accommodation is needed, please contact Kyyba at 248-813-9665
Rewards:
Medical, dental, vision
401k
Term life
Voluntary life and disability insurance
Optional Pre-paid legal plan
Optional Identity theft plan
Optional Medical and dependent FSA
Work-visa sponsorship
Opportunity for advancement
Long-term assignment with opportunity for hire by client
SELECT AWARDS
- An INC 5000 company for 10 years
- Corp! Michigan Economic Bright Spots
- Crain’s Detroit Business Top Staffing Service Companies in Detroit
- TechServe Alliance Excellence Award- IT and Engineering Staffing & Solutions
- Best of MichBusiness winner in HR Wizards & Partnerships
- Metro Detroit Elite Category: Recruitment, Selection & Orientation for 101 Best & Brightest
- 101 Best & Brightest Companies to Work for in Michigan
About Company
Kyyba group of companies are privately held and specialize in staff augmentation, application software and project solutions. In operation for more than 15 years, we have earned an enviable track record and reputation within all the industries we serve. Our unique processes and maturity enables us to understand the needs of the business organizations and provide business solutions that match the real and compelling needs of our customers.
Headquartered in Michigan, Kyyba has multiple office locations and we serve local, regional and national client base consisting of Fortune 500 and middle market companies. Kyyba extends the above solutions and services to a broad spectrum of industry verticals ranging from automotive, insurance, technology, financial, transportation, government and so on.