Skip to content
New Job?Nejo!

Your personal AI career agent

MIMindrift

Freelance Agent Evaluation Engineer(m/w/x)

Stuttgart
from USD 50 / hour
Part-timeFreelanceOn-siteSenior
AI/ML

Building virtual companies and realistic AI development environments. Python development and functional/integration testing experience required. GitHub Actions usage, CI/CD understanding.

Requirements

  • CV in English with English proficiency level indicated
  • Degree in Computer Science, Software Engineering, or related fields
  • 5+ years in software development, primarily Python
  • Experience writing functional and integration tests
  • CI/CD understanding (GitHub Actions user)
  • English proficiency - B2
  • Comfortable reading and reasoning about code across the stack

Tasks

  • Build virtual companies with realistic development environments
  • Assemble and calibrate tasks from intermediate states
  • Craft prompts and define evaluation criteria
  • Ensure tasks are solvable and evaluations are fair
  • Design tasks in isolated developer environments
  • Write tests that accept correct solutions and reject incorrect ones
  • Iterate with AI agents to refine tests
  • Verify tests catch real problems and don't break on valid solutions
  • Review and analyze code written by AI agents
  • Design edge cases and adversarial scenarios
  • Iterate based on feedback from expert QA reviewers
  • Guide and evaluate AI agents' code writing

Work Experience

  • 5 years

Education

  • Bachelor's degree

Languages

  • EnglishAdvanced

Tools & Technologies

  • Python
  • FastAPI
  • pytest
  • async/await
  • subprocess
  • file operations
  • React
  • JavaScript
  • TypeScript
  • Docker
  • Postgres
  • Kafka
  • Redis
  • GitHub Actions
Find the original job posting in its most current version here. Nejo automatically captured this job from the website of Mindrift and processed the information on Nejo with the help of AI for you. Despite careful analysis, some information may be incomplete or inaccurate. Please always verify all details in the original posting! Content and copyrights of the original posting belong to the advertising company.

Like this job?

Beta

Your Career Agent finds similar jobs for you every day.


  • Mindrift

    Freelance Machine Learning Engineer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Data Scientist (Python & SQL) - Freelance AI Trainer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Machine Learning Developer(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Data Science Engineer (Python & SQL)(m/w/x)

    Part-timeFreelanceOn-siteSenior
    Stuttgart
    from USD 58 / hour
  • Mindrift

    Electrical Engineer & Python Expert - Freelance AI Trainer(m/w/x)

    Part-timeFreelanceOn-siteExperienced
    Stuttgart
    from USD 50 / hour
View all 100+ similar jobs

Nejo is an AI – results may be incomplete or contain mistakes