Your personal AI career agent
Freelance Agent Evaluation Engineer(m/w/x)
Building virtual companies and realistic AI development environments. Python development and functional/integration testing experience required. GitHub Actions usage, CI/CD understanding.
Requirements
- CV in English with English proficiency level indicated
- Degree in Computer Science, Software Engineering, or related fields
- 5+ years in software development, primarily Python
- Experience writing functional and integration tests
- CI/CD understanding (GitHub Actions user)
- English proficiency - B2
- Comfortable reading and reasoning about code across the stack
Tasks
- Build virtual companies with realistic development environments
- Assemble and calibrate tasks from intermediate states
- Craft prompts and define evaluation criteria
- Ensure tasks are solvable and evaluations are fair
- Design tasks in isolated developer environments
- Write tests that accept correct solutions and reject incorrect ones
- Iterate with AI agents to refine tests
- Verify tests catch real problems and don't break on valid solutions
- Review and analyze code written by AI agents
- Design edge cases and adversarial scenarios
- Iterate based on feedback from expert QA reviewers
- Guide and evaluate AI agents' code writing
Work Experience
- 5 years
Education
- Bachelor's degree
Languages
- English – Advanced
Tools & Technologies
- Python
- FastAPI
- pytest
- async/await
- subprocess
- file operations
- React
- JavaScript
- TypeScript
- Docker
- Postgres
- Kafka
- Redis
- GitHub Actions
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
Not a perfect match?
- MindriftPart-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour
- Mindrift
Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Machine Learning Developer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Data Science Engineer (Python & SQL)(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Electrical Engineer & Python Expert - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteExperiencedStuttgartfrom USD 50 / hour
Freelance Agent Evaluation Engineer(m/w/x)
Building virtual companies and realistic AI development environments. Python development and functional/integration testing experience required. GitHub Actions usage, CI/CD understanding.
Requirements
- CV in English with English proficiency level indicated
- Degree in Computer Science, Software Engineering, or related fields
- 5+ years in software development, primarily Python
- Experience writing functional and integration tests
- CI/CD understanding (GitHub Actions user)
- English proficiency - B2
- Comfortable reading and reasoning about code across the stack
Tasks
- Build virtual companies with realistic development environments
- Assemble and calibrate tasks from intermediate states
- Craft prompts and define evaluation criteria
- Ensure tasks are solvable and evaluations are fair
- Design tasks in isolated developer environments
- Write tests that accept correct solutions and reject incorrect ones
- Iterate with AI agents to refine tests
- Verify tests catch real problems and don't break on valid solutions
- Review and analyze code written by AI agents
- Design edge cases and adversarial scenarios
- Iterate based on feedback from expert QA reviewers
- Guide and evaluate AI agents' code writing
Work Experience
- 5 years
Education
- Bachelor's degree
Languages
- English – Advanced
Tools & Technologies
- Python
- FastAPI
- pytest
- async/await
- subprocess
- file operations
- React
- JavaScript
- TypeScript
- Docker
- Postgres
- Kafka
- Redis
- GitHub Actions
Like this job?
BetaYour Career Agent finds similar jobs for you every day.
About the Company
Mindrift
Industry
IT
Description
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
Not a perfect match?
- Mindrift
Freelance Machine Learning Engineer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Machine Learning Developer(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Data Science Engineer (Python & SQL)(m/w/x)
Part-timeFreelanceOn-siteSeniorStuttgartfrom USD 58 / hour - Mindrift
Electrical Engineer & Python Expert - Freelance AI Trainer(m/w/x)
Part-timeFreelanceOn-siteExperiencedStuttgartfrom USD 50 / hour