AutoScout24
Principal GenAI Data Scientist(m/w/x)
Full-timeOn-siteExperienced
München
New Job?Nejo!
The AI Job Search Engine
REResaro
Senior AI Scientist, LLM(m/w/x)
München
Full-timeOn-siteSenior
AI/ML
Data Science
Nejo AI Summary
Designing evaluation frameworks for generative AI systems, benchmarking large language models for AI assurance. First principles thinking essential. Work on novel AI evaluation challenges.
Requirements
- Ability to think from first principles
- Leadership by example
- Open and respectful communication
- Ability to thrive in a fast-paced environment
- Willingness to navigate uncertainty
- Extensive experience in deploying LLMs
- Strong experience in evaluating LLMs
- Proven track record with large language datasets
- Excellent written and verbal communication skills
- Solid programming skills in Python
- Passion for applied research in AI
- Published research in generative AI (nice to have)
- Hands-on experience with model explainability tools (nice to have)
- Familiarity with cloud-based platforms (nice to have)
Tasks
- Lead the design and implementation of evaluation frameworks for generative AI systems
- Establish and refine metrics for model quality, including fidelity and bias detection
- Perform technical evaluations and benchmarking on large language models
- Design custom evaluation approaches based on client needs and scientific research
- Develop technical AI evaluation frameworks and tools with the product management team
- Define metrics for evaluating model performance, such as accuracy and coherence
- Curate and manage high-quality datasets for evaluating large language models
- Mentor junior data scientists on best practices for LLM evaluation
- Stay updated on advancements in NLP and LLM evaluation methods
Work Experience
- 5 - 8 years
Education
- Master's degree
Languages
- English – Business Fluent
Tools & Technologies
- Python
- AWS
- GCP
Find the original job posting in its most current version here. Nejo automatically captured this job from the website of Resaro and processed the information on Nejo with the help of AI for you. Despite careful analysis, some information may be incomplete or inaccurate. Please always verify all details in the original posting! Content and copyrights of the original posting belong to the advertising company.
Not a perfect match?
- Agile Robots
Senior LLM / AI Engineer Predictive Maintenance Agent(m/w/x)
Full-timeOn-siteSeniorMünchen - Cobrainer GmbH
Senior AI/ML Platform Engineer(m/w/x)
Full-timeOn-siteSeniorMünchen - SAP
Staff/Principal AI Scientist for Mixture-of-Architecture Foundation Models(m/w/x)
Full-timeOn-siteSeniorGarching bei München, Walldorf, Berlin, Potsdam - Resaro
Solution Architect - Defence and Security(m/w/x)
Full-timeOn-siteSeniorMünchen
REResaro
Senior AI Scientist, LLM(m/w/x)
München
Full-timeOn-siteSenior
AI/ML
Data Science
Nejo AI Summary
Designing evaluation frameworks for generative AI systems, benchmarking large language models for AI assurance. First principles thinking essential. Work on novel AI evaluation challenges.
Requirements
- Ability to think from first principles
- Leadership by example
- Open and respectful communication
- Ability to thrive in a fast-paced environment
- Willingness to navigate uncertainty
- Extensive experience in deploying LLMs
- Strong experience in evaluating LLMs
- Proven track record with large language datasets
- Excellent written and verbal communication skills
- Solid programming skills in Python
- Passion for applied research in AI
- Published research in generative AI (nice to have)
- Hands-on experience with model explainability tools (nice to have)
- Familiarity with cloud-based platforms (nice to have)
Tasks
- Lead the design and implementation of evaluation frameworks for generative AI systems
- Establish and refine metrics for model quality, including fidelity and bias detection
- Perform technical evaluations and benchmarking on large language models
- Design custom evaluation approaches based on client needs and scientific research
- Develop technical AI evaluation frameworks and tools with the product management team
- Define metrics for evaluating model performance, such as accuracy and coherence
- Curate and manage high-quality datasets for evaluating large language models
- Mentor junior data scientists on best practices for LLM evaluation
- Stay updated on advancements in NLP and LLM evaluation methods
Work Experience
- 5 - 8 years
Education
- Master's degree
Languages
- English – Business Fluent
Tools & Technologies
- Python
- AWS
- GCP
Find the original job posting in its most current version here. Nejo automatically captured this job from the website of Resaro and processed the information on Nejo with the help of AI for you. Despite careful analysis, some information may be incomplete or inaccurate. Please always verify all details in the original posting! Content and copyrights of the original posting belong to the advertising company.
About the Company
Resaro
Industry
IT
Description
The company is a global AI Assurance company, pioneering the field of AI testing and evaluation.
Not a perfect match?
- AutoScout24
Principal GenAI Data Scientist(m/w/x)
Full-timeOn-siteExperiencedMünchen - Agile Robots
Senior LLM / AI Engineer Predictive Maintenance Agent(m/w/x)
Full-timeOn-siteSeniorMünchen - Cobrainer GmbH
Senior AI/ML Platform Engineer(m/w/x)
Full-timeOn-siteSeniorMünchen - SAP
Staff/Principal AI Scientist for Mixture-of-Architecture Foundation Models(m/w/x)
Full-timeOn-siteSeniorGarching bei München, Walldorf, Berlin, Potsdam - Resaro
Solution Architect - Defence and Security(m/w/x)
Full-timeOn-siteSeniorMünchen