LLM Application Engineer

Summary

Software Guidance & Assistance, Inc. is a women-owned technology and resource solutions provider that delivers IT staffing and consulting services. The company is seeking an LLM Application Engineer to design, develop, and deploy production-grade applications using large language models, generative AI, and intelligent workflow automation. The role focuses on agent systems, AI orchestration, integrations, evaluation, troubleshooting, optimization, and continuous improvement of production AI solutions.

Responsibilities

• Design, develop, and deploy applications powered by large language models and intelligent workflow automation
• Create agent-based systems that support reasoning, planning, memory management, tool integration, and multi-step task execution
• Build orchestration frameworks that transform AI-generated outputs into dependable, observable, and secure business processes
• Integrate AI solutions with APIs, databases, search platforms, internal services, and third-party systems
• Implement prompt engineering, context management, structured responses, and tool-calling techniques to improve model performance
• Develop evaluation methodologies, benchmarks, and datasets to monitor quality, accuracy, and regression risks
• Troubleshoot issues across the AI stack, including model behavior, orchestration layers, backend systems, and user experience components
• Optimize solutions for performance, scalability, response times, and operational cost
• Partner with product and engineering stakeholders to convert business requirements into practical AI-driven solutions
• Establish best practices for monitoring, experimentation, traceability, and continuous improvement of production AI systems

Skills

• Strong software engineering background with experience building production-grade AI applications
• Hands-on experience with large language models, generative AI technologies, or autonomous agent architectures
• Proficiency in Python development
• Experience designing prompts, workflows, evaluation frameworks, and AI behavior optimization strategies
• Knowledge of LLM platforms, model APIs, and open-source model ecosystems
• Ability to develop clean, maintainable, and scalable production code
• Experience working across multiple layers of AI solutions, including models, systems, and end-user applications
• Strong analytical and problem-solving capabilities in fast-paced and evolving environments
• Experience with AI orchestration frameworks and agent development platforms
• Knowledge of vector databases, retrieval systems, and semantic search technologies
• Experience building backend services, APIs, and distributed systems
• Familiarity with machine learning frameworks such as PyTorch or JAX
• Demonstrated success in iterative product development and continuous improvement initiatives

Qualifications

Must Haves

• Strong software engineering background with experience building production-grade AI applications
• Hands-on experience with large language models, generative AI technologies, or autonomous agent architectures
• Proficiency in Python development
• Experience designing prompts, workflows, evaluation frameworks, and AI behavior optimization strategies
• Knowledge of LLM platforms, model APIs, and open-source model ecosystems
• Ability to develop clean, maintainable, and scalable production code
• Experience working across multiple layers of AI solutions, including models, systems, and end-user applications
• Strong analytical and problem-solving capabilities in fast-paced and evolving environments

Nice to Haves

• Experience with AI orchestration frameworks and agent development platforms
• Knowledge of vector databases, retrieval systems, and semantic search technologies
• Experience building backend services, APIs, and distributed systems
• Familiarity with machine learning frameworks such as PyTorch or JAX
• Demonstrated success in iterative product development and continuous improvement initiatives

Benefits

• Remote work arrangement