Your personal AI career agent
Freelance Agent Evaluation Engineer(m/w/x)
Building virtual companies and AI evaluation environments for tech firms. 5+ years Python full-stack development experience required. Work on diverse AI projects with flexible, remote engagement.
Requirements
- CV in English with English proficiency level indicated
- Degree in Computer Science, Software Engineering, or related fields
- 5+ years in software development, primarily Python
- Full-stack development experience
- Experience building React-based interfaces
- Experience building robust back-end systems
- Experience writing functional and integration tests
- Docker containers
- Familiarity with infrastructure tools
- CI/CD understanding
- GitHub Actions as a user
- Comfortable reading and reasoning about code across the stack
- Deep understanding of model limitations
- Understanding of scenarios revealing model differences
- Writing tests that accept all correct solutions
- Writing tests that reject incorrect solutions
Tasks
- Build virtual companies with realistic development environments
- Create codebases, infrastructure, and contextual elements
- Assemble tasks from intermediate states of virtual companies
- Craft prompts and define evaluation criteria for tasks
- Design isolated environments for developer tasks
- Write tests to validate correct and incorrect solutions
- Iterate with AI agents to refine tests
- Review and analyze code written by AI agents
- Design edge cases and adversarial scenarios
- Iterate based on feedback from expert QA reviewers
- Collaborate with AI to create challenging tasks
Work Experience
- 5 years
Education
- Bachelor's degree
Languages
- English – Advanced
Tools & Technologies
- Python
- FastAPI
- pytest
- async/await
- subprocess
- file operations
- React
- JavaScript
- TypeScript
- Docker
- Postgres
- Kafka
- Redis
- GitHub Actions
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
Not a perfect match?
- MindriftPart-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour
- Mindrift
Freelance Data Science Engineer (Python & SQL)(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Machine Learning Developer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Mechanical Engineer & Python Expert - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteExperiencedStuttgartfrom USD 50 / hour
Freelance Agent Evaluation Engineer(m/w/x)
Building virtual companies and AI evaluation environments for tech firms. 5+ years Python full-stack development experience required. Work on diverse AI projects with flexible, remote engagement.
Requirements
- CV in English with English proficiency level indicated
- Degree in Computer Science, Software Engineering, or related fields
- 5+ years in software development, primarily Python
- Full-stack development experience
- Experience building React-based interfaces
- Experience building robust back-end systems
- Experience writing functional and integration tests
- Docker containers
- Familiarity with infrastructure tools
- CI/CD understanding
- GitHub Actions as a user
- Comfortable reading and reasoning about code across the stack
- Deep understanding of model limitations
- Understanding of scenarios revealing model differences
- Writing tests that accept all correct solutions
- Writing tests that reject incorrect solutions
Tasks
- Build virtual companies with realistic development environments
- Create codebases, infrastructure, and contextual elements
- Assemble tasks from intermediate states of virtual companies
- Craft prompts and define evaluation criteria for tasks
- Design isolated environments for developer tasks
- Write tests to validate correct and incorrect solutions
- Iterate with AI agents to refine tests
- Review and analyze code written by AI agents
- Design edge cases and adversarial scenarios
- Iterate based on feedback from expert QA reviewers
- Collaborate with AI to create challenging tasks
Work Experience
- 5 years
Education
- Bachelor's degree
Languages
- English – Advanced
Tools & Technologies
- Python
- FastAPI
- pytest
- async/await
- subprocess
- file operations
- React
- JavaScript
- TypeScript
- Docker
- Postgres
- Kafka
- Redis
- GitHub Actions
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
About the Company
Mindrift
Industry
IT
Description
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
Not a perfect match?
- Mindrift
Freelance Machine Learning Engineer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Freelance Data Science Engineer (Python & SQL)(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Machine Learning Developer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Mechanical Engineer & Python Expert - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteExperiencedStuttgartfrom USD 50 / hour