Indian English - AI Product Evaluator

Summary

Productive Playhouse is building a talent pool of Indian English speakers for an upcoming project testing and evaluating leading AI chatbots. As an AI Evaluator, you'll play a critical role in shaping and improving the next generation of Generative AI by interacting directly with various AI models to assess their capabilities, safety, and helpfulness.

Responsibilities

• Testing and evaluating AI chatbots or language models through structured conversations, using assigned goals and prompts
• Voice-based interactions with AI models — for projects that involve this, sessions are recorded, and audio may be shared with the client as part of the evaluation deliverable
• Submitting deliverables (write-ups, ratings, screenshots, or recordings) in the format each task specifies

Skills

• Native or expert-level fluency in English, as spoken in India
• Strong English reading, writing, and communication skills - assessment maybe required
• 18 years of age or older
• Hands-on experience using AI models or large language models
• Comfortable with both written and voice-based conversation — and good at asking sharp follow-ups
• Your own smartphone, computer, reliable internet, and comfort with web-based tools
• Able to pass a language proficiency and technical literacy assessment
• Responsive communicator — quick replies move you through onboarding faster
• Able to start within 48 hours

Qualifications

Must Haves

• Native or expert-level fluency in English, as spoken in India
• Strong English reading, writing, and communication skills - assessment maybe required
• 18 years of age or older
• Hands-on experience using AI models or large language models
• Comfortable with both written and voice-based conversation — and good at asking sharp follow-ups
• Your own smartphone, computer, reliable internet, and comfort with web-based tools
• Able to pass a language proficiency and technical literacy assessment
• Responsive communicator — quick replies move you through onboarding faster
• Able to start within 48 hours

Benefits

• Work from anywhere, on your own schedule, alongside any other work you do
• You choose which tasks to take, set your own hours, and are free to work with other clients any time
• Iterative project; work arrives in batches (there may be pauses between them)