Ai Evaluation Expert - Remote

Summary

YO IT CONSULTING is seeking an AI Evaluation Specialist to help train next-generation AI systems. The role involves reviewing AI-generated outputs, identifying improvement opportunities, and collaborating with teams to enhance quality standards.

Responsibilities

• Review and critically assess AI-generated outputs for quality, clarity, usability, and overall user experience
• Identify inconsistencies, weaknesses, and improvement opportunities across diverse content types and visual experiences
• Apply structured evaluation guidelines and provide insightful, nuanced feedback to inform model improvements
• Collaborate with cross-functional teams to define and refine quality standards, scoring criteria, and review processes
• Maintain high accuracy and consistency across evaluations, ensuring reliable and actionable results
• Contribute to the continuous enhancement of evaluation workflows and best practices
• Support the development of training data that enhances AI system performance and reliability

Skills

• Exceptional reading comprehension and acute attention to detail
• Strong observational skills with excellent aesthetic or editorial judgment
• Ability to think critically and deliver consistent, high-quality evaluations in ambiguous scenarios
• Excellent written and verbal communication skills
• Comfort working across various content types and evaluation guidelines
• Strong sense of ownership, reliability, and commitment to delivering high-quality work for the customer's team
• Open-minded approach to learning and quickly adapting to new workflows
• Background in design, UX/UI, creative direction, editing, or content strategy
• Experience reviewing creative, visual, or AI-generated content
• Familiarity with annotation, QA, moderation, or evaluation workflows, as well as working with AI tools or LLMs

Qualifications

Must Haves

• Exceptional reading comprehension and acute attention to detail
• Strong observational skills with excellent aesthetic or editorial judgment
• Ability to think critically and deliver consistent, high-quality evaluations in ambiguous scenarios
• Excellent written and verbal communication skills
• Comfort working across various content types and evaluation guidelines
• Strong sense of ownership, reliability, and commitment to delivering high-quality work for the customer's team
• Open-minded approach to learning and quickly adapting to new workflows

Nice to Haves

• Background in design, UX/UI, creative direction, editing, or content strategy
• Experience reviewing creative, visual, or AI-generated content
• Familiarity with annotation, QA, moderation, or evaluation workflows, as well as working with AI tools or LLMs