Dein persönlicher KI-Karriere-Agent
AI Quality Assurance Engineer(m/w/x)
Validating AI Hub capabilities and supporting decentralized AI use cases. Understanding of prompt engineering and agentic AI testing required. Flexible hybrid work, relocation support.
Anforderungen
- Understanding of prompt/context/harness engineering, intent-based testing, spec-driven development, BDD, codified acceptance criteria, feature flagging, A/B testing
- Min. 3 years experience as engineer, strong advocacy/knowledge of SQA, System Testing in Agentic AI
- Min. 2 years experience, familiarity with evaluation frameworks, behavioural/performance testing, observability platforms (e.g., LangSmith, DeepEval, Ragas, Langfuse, Braintrust, Datadog)
- Experience defining test strategies, test plans, acceptance criteria, validation approaches
- Deep understanding of Generative AI systems (e.g. LLM based applications), agentic AI workflows
- Familiarity with evaluation of GenAI outputs (e.g. hallucination risk, consistency, explainability, robustness)
- Understanding of prompt-based system behaviour, orchestration patterns, AI agents
- Familiarity with data pipelines, APIs, system integration patterns
- Ability to translate functional AI requirements to technically testable specifications
- Experience in Agile delivery environments, cross-functional teams
- Strong analytical and problem-solving skills
- Experience in regulated financial services environments is beneficial
Aufgaben
- Evaluate and validate AI Hub capabilities
- Support decentralized AI use cases
- Define evaluation approaches for AI systems
- Translate AI requirements into testable criteria
- Derive reusable validation methods and best practices
- Promote AI quality assurance methods
- Ensure scalable and compliant AI deployment
- Assess AI system reliability, performance, explainability, robustness
- Implement CI/CD regression gates
- Maintain a golden eval set
- Run evals in CI
- Block rollout on regressions
- Apply best practices in prompt, context, and harness engineering
- Mitigate risks like context rot
- Use spec-driven development frameworks
- Codify clear quality instructions
- Apply feature flagging and A/B testing
- Support continuous improvement of AI solutions
Berufserfahrung
- 3 Jahre
Ausbildung
- Abgeschlossene BerufsausbildungODER
- Bachelor-AbschlussODER
- Master-Abschluss
Sprachen
- Englisch – verhandlungssicher
Tools & Technologien
- LangSmith
- DeepEval
- Ragas
- Langfuse
- Braintrust
- Datadog
- APIs
Benefits
Flexibles Arbeiten
- Flexible work arrangements
- Hybrid model
- Flexible working hours
Betriebliche Altersvorsorge
- Company pension/savings plans
Sonstige Vorteile
- Relocation support
Kinderbetreuung
- Childcare facilities
Attraktive Vergütung
- Company share purchasing plan
Mentale Gesundheitsförderung
- Mental health and wellbeing programs
Firmenfahrrad
- Jobrad bike leasing
Öffi Tickets
- Subvention Jobticket
Karriere- und Weiterentwicklung
- Career opportunities within Allianz Group
Weiterbildungsangebote
- Self-guided learning & development
Gemeinnützige Ausrichtung
- Volunteering time
Gefällt dir diese Stelle?
BetaDein Career Agent findet täglich ähnliche Jobs für dich.
Noch nicht perfekt?
- Allianz Global InvestorsVollzeitmit HomeofficeBerufserfahrenFrankfurt am Main
- Allianz Global Investors
Data Engineer(m/w/x)
Vollzeitmit HomeofficeBerufserfahrenFrankfurt am Main, München - paiqo GmbH
AI Engineer II - Agentic AI Platform(m/w/x)
Vollzeitmit HomeofficeBerufserfahrenPaderborn, Frankfurt am Main - paiqo GmbH
Lead AI Engineer - Agentic AI Platform(m/w/x)
Vollzeitmit HomeofficeSeniorPaderborn, Frankfurt am Main - Deloitte GmbH Wirtschaftsprüfungsgesellschaft
Senior Quality Engineer(m/w/x)
Vollzeitmit HomeofficeSeniorBerlin, Düsseldorf, Frankfurt am Main, Hamburg, Hannover, Köln, München, Nürnberg, Stuttgart
AI Quality Assurance Engineer(m/w/x)
Validating AI Hub capabilities and supporting decentralized AI use cases. Understanding of prompt engineering and agentic AI testing required. Flexible hybrid work, relocation support.
Anforderungen
- Understanding of prompt/context/harness engineering, intent-based testing, spec-driven development, BDD, codified acceptance criteria, feature flagging, A/B testing
- Min. 3 years experience as engineer, strong advocacy/knowledge of SQA, System Testing in Agentic AI
- Min. 2 years experience, familiarity with evaluation frameworks, behavioural/performance testing, observability platforms (e.g., LangSmith, DeepEval, Ragas, Langfuse, Braintrust, Datadog)
- Experience defining test strategies, test plans, acceptance criteria, validation approaches
- Deep understanding of Generative AI systems (e.g. LLM based applications), agentic AI workflows
- Familiarity with evaluation of GenAI outputs (e.g. hallucination risk, consistency, explainability, robustness)
- Understanding of prompt-based system behaviour, orchestration patterns, AI agents
- Familiarity with data pipelines, APIs, system integration patterns
- Ability to translate functional AI requirements to technically testable specifications
- Experience in Agile delivery environments, cross-functional teams
- Strong analytical and problem-solving skills
- Experience in regulated financial services environments is beneficial
Aufgaben
- Evaluate and validate AI Hub capabilities
- Support decentralized AI use cases
- Define evaluation approaches for AI systems
- Translate AI requirements into testable criteria
- Derive reusable validation methods and best practices
- Promote AI quality assurance methods
- Ensure scalable and compliant AI deployment
- Assess AI system reliability, performance, explainability, robustness
- Implement CI/CD regression gates
- Maintain a golden eval set
- Run evals in CI
- Block rollout on regressions
- Apply best practices in prompt, context, and harness engineering
- Mitigate risks like context rot
- Use spec-driven development frameworks
- Codify clear quality instructions
- Apply feature flagging and A/B testing
- Support continuous improvement of AI solutions
Berufserfahrung
- 3 Jahre
Ausbildung
- Abgeschlossene BerufsausbildungODER
- Bachelor-AbschlussODER
- Master-Abschluss
Sprachen
- Englisch – verhandlungssicher
Tools & Technologien
- LangSmith
- DeepEval
- Ragas
- Langfuse
- Braintrust
- Datadog
- APIs
Benefits
Flexibles Arbeiten
- Flexible work arrangements
- Hybrid model
- Flexible working hours
Betriebliche Altersvorsorge
- Company pension/savings plans
Sonstige Vorteile
- Relocation support
Kinderbetreuung
- Childcare facilities
Attraktive Vergütung
- Company share purchasing plan
Mentale Gesundheitsförderung
- Mental health and wellbeing programs
Firmenfahrrad
- Jobrad bike leasing
Öffi Tickets
- Subvention Jobticket
Karriere- und Weiterentwicklung
- Career opportunities within Allianz Group
Weiterbildungsangebote
- Self-guided learning & development
Gemeinnützige Ausrichtung
- Volunteering time
Gefällt dir diese Stelle?
BetaDein Career Agent findet täglich ähnliche Jobs für dich.
Über das Unternehmen
Allianz Global Investors
Branche
FinancialServices
Beschreibung
Allianz Global Investors is a leading global active asset manager focused on long-term value creation and sustainability.
Noch nicht perfekt?
- Allianz Global Investors
DevSecOps AI Engineer(m/w/x)
Vollzeitmit HomeofficeBerufserfahrenFrankfurt am Main - Allianz Global Investors
Data Engineer(m/w/x)
Vollzeitmit HomeofficeBerufserfahrenFrankfurt am Main, München - paiqo GmbH
AI Engineer II - Agentic AI Platform(m/w/x)
Vollzeitmit HomeofficeBerufserfahrenPaderborn, Frankfurt am Main - paiqo GmbH
Lead AI Engineer - Agentic AI Platform(m/w/x)
Vollzeitmit HomeofficeSeniorPaderborn, Frankfurt am Main - Deloitte GmbH Wirtschaftsprüfungsgesellschaft
Senior Quality Engineer(m/w/x)
Vollzeitmit HomeofficeSeniorBerlin, Düsseldorf, Frankfurt am Main, Hamburg, Hannover, Köln, München, Nürnberg, Stuttgart