Skip to content
New Job?Nejo!

Your personal AI career agent

MIMindrift

Freelance Agent Evaluation Engineer(m/w/x)

Stuttgart
from USD 50 / hour
Part-timeFreelanceOn-siteSenior
AI/ML

Building virtual companies and AI evaluation environments for tech firms. 5+ years Python full-stack development experience required. Work on diverse AI projects with flexible, remote engagement.

Requirements

  • CV in English with English proficiency level indicated
  • Degree in Computer Science, Software Engineering, or related fields
  • 5+ years in software development, primarily Python
  • Full-stack development experience
  • Experience building React-based interfaces
  • Experience building robust back-end systems
  • Experience writing functional and integration tests
  • Docker containers
  • Familiarity with infrastructure tools
  • CI/CD understanding
  • GitHub Actions as a user
  • Comfortable reading and reasoning about code across the stack
  • Deep understanding of model limitations
  • Understanding of scenarios revealing model differences
  • Writing tests that accept all correct solutions
  • Writing tests that reject incorrect solutions

Tasks

  • Build virtual companies with realistic development environments
  • Create codebases, infrastructure, and contextual elements
  • Assemble tasks from intermediate states of virtual companies
  • Craft prompts and define evaluation criteria for tasks
  • Design isolated environments for developer tasks
  • Write tests to validate correct and incorrect solutions
  • Iterate with AI agents to refine tests
  • Review and analyze code written by AI agents
  • Design edge cases and adversarial scenarios
  • Iterate based on feedback from expert QA reviewers
  • Collaborate with AI to create challenging tasks

Work Experience

  • 5 years

Education

  • Bachelor's degree

Languages

  • EnglishAdvanced

Tools & Technologies

  • Python
  • FastAPI
  • pytest
  • async/await
  • subprocess
  • file operations
  • React
  • JavaScript
  • TypeScript
  • Docker
  • Postgres
  • Kafka
  • Redis
  • GitHub Actions
Find the original job posting in its most current version here. Nejo automatically captured this job from the website of Mindrift and processed the information on Nejo with the help of AI for you. Despite careful analysis, some information may be incomplete or inaccurate. Please always verify all details in the original posting! Content and copyrights of the original posting belong to the advertising company.

  • Mindrift

    Freelance Machine Learning Engineer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Freelance Data Science Engineer (Python & SQL)(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Machine Learning Developer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Electrical Engineer & Python Expert - Freelance AI Trainer(m/w/x)

    Part-timeFreelanceOn-siteExperienced
    Stuttgart
    from USD 50 / hour
View all 100+ similar jobs

Nejo is an AI – results may be incomplete or contain mistakes