Associate Product Manager, AI Tools

Summary

Internet Brands is seeking an Associate Product Manager, AI Products to help build, manage, evaluate, and optimize AI-powered product experiences across various platforms. The role involves working closely with cross-functional teams to define AI behavior, measure quality, and improve AI performance through disciplined iteration.

Responsibilities

• Support the development and optimization of AI-powered product features, including conversational interfaces, search/Q&A experiences, content generation workflows, recommendation experiences, personalization, and intelligent workflow tools
• Create, manage, document, and iterate on prompts used across AI product experiences, including system prompts, user-facing prompt flows, structured task prompts, and workflow-specific instructions
• Help define prompt management practices, including versioning, documentation, use-case mapping, change logs, testing workflows, and approval processes for prompt updates
• Partner with senior product leaders, engineering, ML/data science, UX, and content teams to translate product goals into clear AI behavior requirements, prompt specifications, evaluation plans, and launch criteria
• Define evaluation criteria for prompt changes, including accuracy, relevance, completeness, tone, clarity, hallucination risk, groundedness, safety, compliance, consistency, latency, and user task success
• Build and maintain prompt evaluation frameworks that measure quality before and after changes, using human review, test sets, rubric-based scoring, structured QA, user feedback, and product analytics
• Help design experiments to compare prompt variants and measure their impact on AI system performance, user experience, engagement, task completion, and quality metrics
• Review AI outputs across common, edge-case, and high-risk scenarios to identify failure modes, quality gaps, confusing responses, unsafe outputs, or opportunities for improvement
• Support the creation of golden test sets, evaluation rubrics, sample queries, expected answer patterns, and benchmark scenarios for ongoing AI quality monitoring
• Analyze prompt performance using dashboards, SQL-based reporting, qualitative feedback, annotation results, and product usage data to identify where prompts and AI workflows should be improved
• Collaborate with content, legal, compliance, and editorial teams to ensure AI outputs are accurate, trustworthy, transparent, and aligned with company standards
• Document product requirements, AI behavior expectations, evaluation results, prompt decisions, experiment learnings, and recommendations so teams can make informed product and technical decisions
• Support rapid iteration cycles from prototype to pilot to production by helping teams test AI behavior, measure results, and prioritize improvements

Skills

• 1–3 years of product management, AI product, prompt engineering, product operations, research, analytics, or related experience
• Hands-on experience working with LLM-powered products, conversational agents, prompt systems, generative AI workflows, AI copilots, or intelligent automation tools
• Experience writing, testing, refining, or managing prompts for multi-turn AI experiences or structured AI workflows
• Strong ability to evaluate AI outputs and identify quality issues related to accuracy, relevance, completeness, tone, consistency, safety, hallucination risk, or user usefulness
• Familiarity with AI evaluation methods such as rubric-based scoring, human review, golden test sets, prompt comparison testing, grounding checks, safety review, and feedback loop design
• Strong analytical skills, including comfort using product data, dashboards, experiments, user feedback, or structured QA results to recommend improvements
• Ability to translate ambiguous user and business problems into clear product requirements, AI behavior expectations, prompt specifications, and measurable success criteria
• Experience collaborating cross-functionally with engineering, data science, UX, content, legal/compliance, customer-facing, or business teams
• Strong written communication skills, with the ability to document prompt behavior, evaluation criteria, product decisions, and experiment results clearly
• Curious, detail-oriented, and comfortable working in fast-moving AI product environments where product requirements and technical capabilities evolve quickly
• Experience with enterprise AI products, workflow automation, search/Q&A products, recommendation systems, self-serve platforms, or data-rich user experiences
• Experience supporting pilots, demos, user testing, customer feedback sessions, A/B tests, or product experiments
• Exposure to SQL, analytics dashboards, product instrumentation, structured QA workflows, or annotation tools
• Experience improving AI behavior through prompt refinement, grounding, model evaluation, safety testing, or structured feedback collection
• Familiarity with retrieval-augmented generation, prompt versioning, AI quality scorecards, human-in-the-loop review, or AI response monitoring
• Background or coursework in cognitive science, computer science, human-computer interaction, data analysis, linguistics, research methods, or a related field

Qualifications

Must Haves

• 1–3 years of product management, AI product, prompt engineering, product operations, research, analytics, or related experience
• Hands-on experience working with LLM-powered products, conversational agents, prompt systems, generative AI workflows, AI copilots, or intelligent automation tools
• Experience writing, testing, refining, or managing prompts for multi-turn AI experiences or structured AI workflows
• Strong ability to evaluate AI outputs and identify quality issues related to accuracy, relevance, completeness, tone, consistency, safety, hallucination risk, or user usefulness
• Familiarity with AI evaluation methods such as rubric-based scoring, human review, golden test sets, prompt comparison testing, grounding checks, safety review, and feedback loop design
• Strong analytical skills, including comfort using product data, dashboards, experiments, user feedback, or structured QA results to recommend improvements
• Ability to translate ambiguous user and business problems into clear product requirements, AI behavior expectations, prompt specifications, and measurable success criteria
• Experience collaborating cross-functionally with engineering, data science, UX, content, legal/compliance, customer-facing, or business teams
• Strong written communication skills, with the ability to document prompt behavior, evaluation criteria, product decisions, and experiment results clearly
• Curious, detail-oriented, and comfortable working in fast-moving AI product environments where product requirements and technical capabilities evolve quickly

Nice to Haves

• Experience with enterprise AI products, workflow automation, search/Q&A products, recommendation systems, self-serve platforms, or data-rich user experiences
• Experience supporting pilots, demos, user testing, customer feedback sessions, A/B tests, or product experiments
• Exposure to SQL, analytics dashboards, product instrumentation, structured QA workflows, or annotation tools
• Experience improving AI behavior through prompt refinement, grounding, model evaluation, safety testing, or structured feedback collection
• Familiarity with retrieval-augmented generation, prompt versioning, AI quality scorecards, human-in-the-loop review, or AI response monitoring
• Background or coursework in cognitive science, computer science, human-computer interaction, data analysis, linguistics, research methods, or a related field

Benefits

• Health insurance options such as medical, dental, and vision coverage
• Flexible spending accounts (FSA) for medical and dependent care
• Short-term and long-term disability insurance
• Life and AD&D insurance
• 401(k) retirement savings plan with a company match
• Paid time off (PTO)
• Paid holidays
• Commuter benefits
• Access to our Employee Assistance Program (EAP) and well-being coaching services
• Voluntary benefits such as home, auto and pet insurance
• Discounted legal and financial services