Member of Technical Staff — Model Optimization and Inference (New Grad)

Summary

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence. They are seeking an early-career engineer to optimize model inference for real-time conversations, focusing on reducing latency and enhancing performance across their AI systems.

Responsibilities

• Contribute to end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
• Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
• Work with inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) and extend them for our specific workloads
• Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
• Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
• Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
• Apply quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
• Work closely with research and infrastructure to ensure new models ship with optimized serving from day one

Skills

• BS, MS, or PhD in CS, ML, or a related field — completed or in the final stretch
• Strong fundamentals in LLM inference or ML systems — KV caching, memory layout, attention kernels, batching, or serving — picked up through coursework, research, internships, or open-source
• Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM, or similar) — even at a research or hobby level
• Strong Python and PyTorch skills; familiarity with CUDA or Triton is a significant plus
• A systematic approach to profiling and optimization — you measure first, then optimize
• Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques
• Internship or research experience with LLM inference, ML systems, or model serving
• Contributions to open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.)
• CUDA / Triton kernel work, even at a research or hobby scale
• Publications or research projects in MLSys, model compression, or inference optimization
• Familiarity with multimodal or streaming inference architectures
• Experience with hard latency SLAs in any real-time system

Qualifications

Must Haves

• BS, MS, or PhD in CS, ML, or a related field — completed or in the final stretch
• Strong fundamentals in LLM inference or ML systems — KV caching, memory layout, attention kernels, batching, or serving — picked up through coursework, research, internships, or open-source
• Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM, or similar) — even at a research or hobby level
• Strong Python and PyTorch skills; familiarity with CUDA or Triton is a significant plus
• A systematic approach to profiling and optimization — you measure first, then optimize
• Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques

Nice to Haves

• Internship or research experience with LLM inference, ML systems, or model serving
• Contributions to open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.)
• CUDA / Triton kernel work, even at a research or hobby scale
• Publications or research projects in MLSys, model compression, or inference optimization
• Familiarity with multimodal or streaming inference architectures
• Experience with hard latency SLAs in any real-time system

Benefits

• Visa sponsorship: We sponsor visas (O-1, H-1B, green card) from day one.
• HSA plan with ~$2,000 in annual company contributions — roughly 2x what most big tech companies put in.
• 15 days of PTO plus public holidays, and we close the office for a full week at year-end.
• Lunch, drinks, and snacks on us every workday — the small thing that quietly makes the day better.
• We help cover the cost of getting to the office.
• 401(k): In the works.