Software Engineer, Inference AI/ML

Summary

CoreWeave is The Essential Cloud for AI™, delivering a platform that enables innovators to build and scale AI. The role involves joining the Inference team to implement features and improvements for model serving on the GPU platform, with a focus on latency, reliability, and cost.

Responsibilities

• Implement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve)
• Write tests, code comments, and short design docs; participate in code reviews
• Add basic metrics and dashboards; assist with alarms and runbooks
• Follow on-call runbooks and learn incident response in a guided rotation
• Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance

Skills

• BS/MS in CS, EE, or related field, or equivalent practical experience
• Foundations in data structures, algorithms, and networked services
• Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics
• Exposure to containers and Kubernetes (coursework or projects welcome)
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming)
• Internship or project that deployed a microservice or ML inference demo
• Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus
• Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling

Qualifications

Must Haves

• BS/MS in CS, EE, or related field, or equivalent practical experience
• Foundations in data structures, algorithms, and networked services
• Experience with Python or Go (C++ a plus) and Linux fundamentals; Git/CI basics
• Exposure to containers and Kubernetes (coursework or projects welcome)
• Curiosity about GPU inference concepts (micro-batching, KV cache, streaming)

Nice to Haves

• Internship or project that deployed a microservice or ML inference demo
• Coursework/research with PyTorch or TensorFlow; simple CUDA projects a plus
• Familiarity with Grafana/Prometheus/OpenTelemetry or similar tooling

Benefits

• Discretionary bonus
• Equity awards
• Comprehensive benefits program (all based on eligibility)
• Medical, dental, and vision insurance - 100% paid for by CoreWeave
• Company-paid Life Insurance
• Voluntary supplemental life insurance
• Short and long-term disability insurance
• Flexible Spending Account
• Health Savings Account
• Tuition Reimbursement
• Ability to Participate in Employee Stock Purchase Program (ESPP)
• Mental Wellness Benefits through Spring Health
• Family-Forming support provided by Carrot
• Paid Parental Leave
• Flexible, full-service childcare support with Kinside
• 401(k) with a generous employer match
• Flexible PTO
• Catered lunch each day in our office and data center locations
• A casual work environment
• A work culture focused on innovative disruption
• Hybrid work environment
• Remote work may be considered for candidates located more than 30 miles from an office, based on role requirements for specialized skill sets
• New hires will be invited to attend onboarding at one of our hubs within their first month
• Teams also gather quarterly to support collaboration