AI Engineer

Summary

PlanSource is a leading cloud-based provider of benefits administration technology and services. They are seeking an AI Engineer to design, build, deploy, and support AI-powered products that enhance user experiences in a regulated SaaS environment.

Responsibilities

• Develop and maintain backend services powering AI-driven user experiences
• Build and optimize agentic workflows that support complex business processes
• Implement prompt engineering, content filtering, and response quality improvements using large language models
• Integrate secure handling of sensitive information through masking, tokenization, and other privacy-preserving techniques
• Support conversational experiences, workflow automation, human escalation scenarios, and streaming responses
• Implement feedback mechanisms, session management, and audit capabilities
• Build and enhance document ingestion pipelines for structured and unstructured content
• Design and optimize chunking, embedding, retrieval, and ranking strategies
• Improve semantic search quality through evaluation and experimentation
• Implement secure multi-tenant access controls and data isolation
• Support service-to-service integrations and authentication across AI components
• Deploy and maintain observability tooling for AI applications, including tracing, latency analysis, cost monitoring, and prompt debugging
• Build evaluation frameworks to measure retrieval quality, response quality, and task completion success rates
• Investigate production issues involving model behavior, retrieval accuracy, latency, and integrations
• Develop monitoring, alerting, operational dashboards, and support runbooks
• Contribute to incident response, problem management, and continuous improvement activities
• Containerize and deploy AI services on cloud-native infrastructure
• Manage CI/CD pipelines, container registries, and deployment automation
• Support cloud infrastructure, storage services, and security controls
• Address security vulnerabilities and technical debt in AI-related services
• Participate in architecture reviews and operational readiness reviews
• Partner closely with product managers to translate business requirements into technical solutions
• Collaborate regularly with AI engineers, architects, and engineering teams across the United States and India
• Work effectively within a blended delivery model where architecture, design, implementation, and support responsibilities may be shared across global teams
• Participate in engineering design discussions, technical reviews, and solution delivery planning
• Contribute to engineering standards, documentation, and operational excellence practices
• Participate in agile planning, sprint execution, and cross-functional collaboration

Skills

• Demonstrated experience taking software or AI-powered solutions from development through production deployment and ongoing operational support
• Practical experience building, delivering, and supporting production software or AI-powered solutions in a professional environment
• Strong Python development skills, including APIs, backend services, and asynchronous programming
• Experience building applications using large language models such as OpenAI or equivalent platforms
• Experience with Retrieval-Augmented Generation, embeddings, vector databases, and document processing pipelines
• Experience with agent orchestration frameworks such as LangGraph, LangChain, or similar technologies
• Experience deploying and supporting cloud-native applications in production environments
• Working knowledge of Kubernetes, Docker, CI/CD pipelines, and infrastructure automation
• Experience working with relational databases and SQL
• Strong troubleshooting and problem-solving skills across distributed systems
• Strong collaboration and communication skills, including experience working across geographically distributed teams
• Experience with observability platforms such as Langfuse, LangSmith, Arize Phoenix, or similar tools
• Experience implementing AI evaluation, testing, or quality measurement frameworks
• Familiarity with PII handling, masking, tokenization, and privacy-preserving architectures
• Experience building multi-tenant SaaS applications
• Understanding of compliance frameworks such as HIPAA, SOC 2, GDPR, or CCPA
• Experience with OAuth 2.0 and modern authentication patterns
• Familiarity with AWS services including EKS, S3, IAM, CloudWatch, and container-based deployments
• Exposure to data engineering, ETL pipelines, or analytics platforms

Qualifications

Must Haves

• Demonstrated experience taking software or AI-powered solutions from development through production deployment and ongoing operational support
• Practical experience building, delivering, and supporting production software or AI-powered solutions in a professional environment
• Strong Python development skills, including APIs, backend services, and asynchronous programming
• Experience building applications using large language models such as OpenAI or equivalent platforms
• Experience with Retrieval-Augmented Generation, embeddings, vector databases, and document processing pipelines
• Experience with agent orchestration frameworks such as LangGraph, LangChain, or similar technologies
• Experience deploying and supporting cloud-native applications in production environments
• Working knowledge of Kubernetes, Docker, CI/CD pipelines, and infrastructure automation
• Experience working with relational databases and SQL
• Strong troubleshooting and problem-solving skills across distributed systems
• Strong collaboration and communication skills, including experience working across geographically distributed teams

Nice to Haves

• Experience with observability platforms such as Langfuse, LangSmith, Arize Phoenix, or similar tools
• Experience implementing AI evaluation, testing, or quality measurement frameworks
• Familiarity with PII handling, masking, tokenization, and privacy-preserving architectures
• Experience building multi-tenant SaaS applications
• Understanding of compliance frameworks such as HIPAA, SOC 2, GDPR, or CCPA
• Experience with OAuth 2.0 and modern authentication patterns
• Familiarity with AWS services including EKS, S3, IAM, CloudWatch, and container-based deployments
• Exposure to data engineering, ETL pipelines, or analytics platforms

Benefits

• Comprehensive health coverage with multiple medical plan options - all covering 100% of in-network preventive care.
• Employer-fundedHealth Savings Account (HSA)- up to$1,000 annuallyfor family coverage.
• Dental & Vision planswith 100% coverage for routine dental careand$250 vision frame allowance, plusemployee-onlyvision premiums at$0.
• 401(k) with immediate vestinganda 50% company match up to 6%of contributions.
• Generous paid parental leave, adoptionassistance, and fertility benefits.
• Flexible PTO, paid holidays, a strong culture ofwork-lifebalance and Flex Fridays in the summer.
• Mental health & wellbeingsupport, including Employee Assistance Program (EAP), movement and wellness resources.
• Rewards and recognition programsthat celebrate employees through peer recognition, awards, and quarterly recognition initiatives.