AI Quality Assurance and Security Testing Specialist

Summary

IS3 Solutions is seeking an AI Quality Assurance and Security Testing Specialist to evaluate, test, and validate enterprise AI systems, agentic workflows, and LLM-based applications. The role will design and execute AI QA and security test plans, assess model behavior and detection logic, automate testing workflows, document findings, and collaborate with engineering and security teams to improve AI system resilience.

Responsibilities

• Develop and execute AI QA test plans for agentic AI systems, LLM applications, and autonomous workflows
• Perform security testing including prompt injection attempts, jailbreak simulations, model manipulation, and adversarial input testing
• Validate AI detection logic for accuracy, reliability, and operational value across SIEM and XDR platforms
• Evaluate AI model behavior under stress, edge cases, and adversarial conditions to identify vulnerabilities
• Test AI system integrations across cloud services, APIs, identity systems, endpoints, and enterprise applications
• Automate AI QA workflows using Python, test automation frameworks, and AI evaluation tools
• Document test results, methodologies, and recommendations in a clear and maintainable format
• Collaborate with SOC, engineering, and AI platform teams to improve AI security posture and detection quality
• Research emerging AI threats and testing techniques to ensure continuous improvement of QA processes

Skills

• * Bachelor's or Master's degree in Computer Science, Cybersecurity, AI, Data Science, or a related discipline
• * 3–5+ years of experience in QA engineering, security testing, AI evaluation, or cybersecurity
• * Strong understanding of LLMs, agentic AI, RAG, and modern AI architectures
• * Hands-on experience with test automation, Python, and AI evaluation frameworks
• * Experience with Microsoft Sentinel, Microsoft Defender, and enterprise security monitoring tools
• * Knowledge of identity, endpoint, network, and API security principles
• * Experience conducting security tests and validating detection logic
• * Strong analytical, documentation, and communication skills
• * Experience with Microsoft Copilot, Azure AI Foundry, Azure OpenAI Service, or enterprise AI platforms
• * Familiarity with AI security frameworks such as OWASP LLM Top 10, MITRE ATLAS, and NIST AI RMF
• * Experience performing AI-red team testing, model robustness evaluation, or adversarial QA
• * Certifications such as SC-200, SC-100, GIAC, AI-related certifications, or QA/security testing credentials

Qualifications

Must Haves

• * Bachelor's or Master's degree in Computer Science, Cybersecurity, AI, Data Science, or a related discipline
• * 3–5+ years of experience in QA engineering, security testing, AI evaluation, or cybersecurity
• * Strong understanding of LLMs, agentic AI, RAG, and modern AI architectures
• * Hands-on experience with test automation, Python, and AI evaluation frameworks
• * Experience with Microsoft Sentinel, Microsoft Defender, and enterprise security monitoring tools
• * Knowledge of identity, endpoint, network, and API security principles
• * Experience conducting security tests and validating detection logic
• * Strong analytical, documentation, and communication skills

Nice to Haves

• * Experience with Microsoft Copilot, Azure AI Foundry, Azure OpenAI Service, or enterprise AI platforms
• * Familiarity with AI security frameworks such as OWASP LLM Top 10, MITRE ATLAS, and NIST AI RMF
• * Experience performing AI-red team testing, model robustness evaluation, or adversarial QA
• * Certifications such as SC-200, SC-100, GIAC, AI-related certifications, or QA/security testing credentials

Benefits

• Remote work arrangement