Description

In this role, you will be at the forefront of machine learning, designing data pipelines and collaborating with researchers to ensure high-quality datasets. Your work will directly impact the development of innovative multimodal agents, making a real difference in AI initiatives.

Requirements

• •Strong software engineering skills in Python
• •Practical experience with prompt engineering
• •Experience with ML data workflows
• •Hands-on experience with data pipelines for ML training
• •Familiarity with annotation tooling and data collection
• •Understanding of ML training requirements
• •Experience loading and writing large datasets to/from cloud infrastructure
• •Strong communication skills
• •Collaborative approach and ownership
• •Experience with preference data collection for RLHF
• •Familiarity with multimodal data
• •Experience building synthetic data generation pipelines
• •Background in data quality metrics and monitoring
• •Contributions to dataset releases or benchmarks

Tasks

• •Design and build data pipelines for agent training
• •Collect, filter, deduplicate, format, and version data from multimodal sources
• •Develop tools for dataset construction, including annotation workflows and synthetic data generation
• •Own data quality by building validation frameworks and monitoring for drift and contamination
• •Create evaluation datasets and benchmarks in collaboration with researchers
• •Build and maintain infrastructure for efficient data loading, storage, and retrieval
• •Collaborate with research scientists to translate research requirements into data specifications
• •Document datasets thoroughly, including provenance and intended use cases
• •Profile and optimize research code for training and inference efficiency
• •Implement comprehensive test coverage for data pipelines and ML workflows
• •Elevate codebase quality through code reviews and refactoring
• •Contribute to team roadmaps by identifying data bottlenecks and proposing solutions

Tools & Technologies

Benefits