Summary
Cohere is on a mission to scale intelligence for humanity by training and deploying frontier models for AI systems. The Member of Technical Staff will contribute to model training pipelines, improve training infrastructure, and support the research effort by writing production code and implementing tools for training efficiency.
Responsibilities
- Design and write high-performant and scalable software for training
- Improve our training setup from an infrastructure and codebase performance standpoint
- Craft and implement tools to speed up our training cycles and improve the overall efficacy of our training infrastructure
- Research, implement, and experiment with ideas on our supercompute and data infrastructure
- Learn from and work with the best researchers in the field
Skills
- Extremely strong software engineering skills
- Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR
- Experience with distributed training infrastructures (Kubernetes, Slurm) and associated frameworks (Ray)
- Experience using large-scale distributed training strategies
- Hands on experience on training large model at scale and having contributed to the tooling and/or setup of the training infrastructure
- Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
Qualifications
Must Haves
- Extremely strong software engineering skills
- Proficiency in Python and related ML frameworks such as JAX, Pytorch and XLA/MLIR
- Experience with distributed training infrastructures (Kubernetes, Slurm) and associated frameworks (Ray)
- Experience using large-scale distributed training strategies
- Hands on experience on training large model at scale and having contributed to the tooling and/or setup of the training infrastructure
Nice to Haves
- Bonus: paper at top-tier venues (such as NeurIPS, ICML, ICLR, AIStats, MLSys, JMLR, AAAI, Nature, COLING, ACL, EMNLP)
Benefits
- An open and inclusive culture and work environment
- Work closely with a team on the cutting edge of AI research
- Weekly lunch stipend, in-office lunches & snacks
- Full health and dental benefits, including a separate budget to take care of your mental health
- 100% Parental Leave top-up for up to 6 months
- Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- 6 weeks of vacation (30 working days!)