Your personal AI career agent
Member of Technical Staff - Large scale data infrastructure(m/w/x)
At a generative deep learning firm, building scalable infrastructure for massive image/video datasets, optimizing data retrieval for GPU utilization. Experience building reliable, scalable data loaders and deep cloud object storage knowledge required. High-performance computing environment for AI research.
Requirements
- Management of large-scale object storage
- Strong proficiency in Python
- Experience with various file systems for data manipulation
- Experience building reliable and scalable data loaders
- Deep knowledge of cloud object storage
- Hands-on familiarity with S3 and Azure Blob Storage
- Experience with cloud platforms (AWS, GCP, or Azure)
- Experience with Slurm/HPC environments
- Creation and management of PB-scale storage infrastructure
- Experience with large scale image and video data
Tasks
- Develop and maintain scalable infrastructure for massive image and video datasets
- Optimize data retrieval for efficient GPU utilization during training runs
- Build tools to manage datasets effectively
- Coordinate data transfers from licensing partners
- Ensure efficient use of object storage
Work Experience
- approx. 4 - 6 years
Education
- Bachelor's degreeOR
- Master's degree
Languages
- English – Business Fluent
Tools & Technologies
- Python
- S3
- Azure Blob Storage
- AWS
- GCP
- Azure
- Slurm
- HPC
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
Not a perfect match?
- Black Forest LabsFull-timeOn-siteExperiencedFreiburg im Breisgau
- Prior Labs
Senior ML Infrastructure Engineer(m/w/x)
Full-timeOn-siteSeniorFreiburg im Breisgau, Berlin - Prior Labs
ML Engineer, Cloud Platform(m/w/x)
Full-timeOn-siteExperiencedBerlin, Freiburg im Breisgaufrom 140,000 / year - Prior Labs
ML Engineer, Foundation Model(m/w/x)
Full-timeOn-siteExperiencedBerlin, Freiburg im Breisgaufrom 120,000 / year - Prior Labs
Research Scientist Intern (PhD)(m/w/x)
Full-timeInternshipOn-siteBerlin, Freiburg im Breisgau
Member of Technical Staff - Large scale data infrastructure(m/w/x)
At a generative deep learning firm, building scalable infrastructure for massive image/video datasets, optimizing data retrieval for GPU utilization. Experience building reliable, scalable data loaders and deep cloud object storage knowledge required. High-performance computing environment for AI research.
Requirements
- Management of large-scale object storage
- Strong proficiency in Python
- Experience with various file systems for data manipulation
- Experience building reliable and scalable data loaders
- Deep knowledge of cloud object storage
- Hands-on familiarity with S3 and Azure Blob Storage
- Experience with cloud platforms (AWS, GCP, or Azure)
- Experience with Slurm/HPC environments
- Creation and management of PB-scale storage infrastructure
- Experience with large scale image and video data
Tasks
- Develop and maintain scalable infrastructure for massive image and video datasets
- Optimize data retrieval for efficient GPU utilization during training runs
- Build tools to manage datasets effectively
- Coordinate data transfers from licensing partners
- Ensure efficient use of object storage
Work Experience
- approx. 4 - 6 years
Education
- Bachelor's degreeOR
- Master's degree
Languages
- English – Business Fluent
Tools & Technologies
- Python
- S3
- Azure Blob Storage
- AWS
- GCP
- Azure
- Slurm
- HPC
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
About the Company
Black Forest Labs
Industry
IT
Description
The company advances generative deep learning for media, creating models that transform ideas into images and videos.
Not a perfect match?
- Black Forest Labs
Member of Technical Staff - Training Cluster Engineer(m/w/x)
Full-timeOn-siteExperiencedFreiburg im Breisgau - Prior Labs
Senior ML Infrastructure Engineer(m/w/x)
Full-timeOn-siteSeniorFreiburg im Breisgau, Berlin - Prior Labs
ML Engineer, Cloud Platform(m/w/x)
Full-timeOn-siteExperiencedBerlin, Freiburg im Breisgaufrom 140,000 / year - Prior Labs
ML Engineer, Foundation Model(m/w/x)
Full-timeOn-siteExperiencedBerlin, Freiburg im Breisgaufrom 120,000 / year - Prior Labs
Research Scientist Intern (PhD)(m/w/x)
Full-timeInternshipOn-siteBerlin, Freiburg im Breisgau