Description

In this role, you will enhance fine-tuning methodologies for cutting-edge language models while researching and implementing advanced optimization techniques. Your work will involve re-implementing state-of-the-art architectures to ensure efficient and high-quality model training.

Requirements

• •Profound understanding of theoretical foundations of machine learning and reinforcement learning
• •Deep expertise in modern deep learning for language processing and generation
• •Substantial experience with training large models on multiple computational nodes
• •Reasonable understanding of performance aspects of large neural network training
• •Strong software engineering skills
• •Deep experience with modern deep learning frameworks
• •Proficiency in contemporary software engineering approaches
• •Strong communication and leadership abilities
• •Previous experience working with language models or similar NLP technologies
• •Familiarity with important ideas in LLM space
• •Track record of building and delivering products in a dynamic startup-like environment
• •Strong engineering skills in developing large distributed systems
• •Open-source projects that showcase engineering prowess
• •Excellent command of the English language and superior communication skills

Tasks

• •Enhance fine-tuning methodologies for LLMs
• •Focus on model quality and training efficiency
• •Research advanced inference optimization techniques
• •Implement speculative decoding and quantization
• •Train large-scale draft models
• •Re-implement open-source LLM architectures in JAX

Tools & Technologies

Benefits