Research Engineer / Research Scientist, Pre-training(m/w/x)

Description

Developing multimodal architectures and data processing optimizers for large-scale model training. Expertise in Python, deep learning, and ML accelerators required. Optional equity donation matching and generous parental leave.

Requirements

• •Degree in Computer Science or related
• •Strong software engineering skills
• •Expertise in Python and deep learning
• •Experience with large-scale ML systems
• •Familiarity with ML Accelerators and Kubernetes
• •Problem-solving skills and results-oriented mindset
• •Excellent communication and collaboration skills
• •Significant software engineering experience
• •Ability to balance research and engineering
• •Willingness to support the team
• •Enjoyment of pair programming
• •Eagerness to learn machine learning
• •Enthusiasm for large-scale AI research
• •Ambitious goals for AI safety

Tasks

• •Develop next-generation large language models
• •Enhance multimodal capabilities for non-text interactions
• •Build safe, steerable, and trustworthy AI systems
• •Research and implement model architectures and algorithms
• •Develop data processing solutions and optimizers
• •Lead small research projects independently
• •Collaborate with team members on large-scale initiatives
• •Design and run scientific experiments on LLMs
• •Analyze experiments to advance model understanding
• •Optimize and scale training infrastructure for efficiency
• •Develop dev tooling to enhance team productivity
• •Contribute to low-level optimizations and high-level design
• •Optimize throughput of novel attention mechanisms
• •Propose and experimentally compare Transformer variants
• •Prepare large-scale datasets for model consumption
• •Scale distributed training to thousands of accelerators
• •Design fault tolerance strategies for training infrastructure
• •Create interactive visualizations of model internals

Tools & Technologies

Benefits