Skip to content
New Job?Nejo!

Your personal AI career agent

Back
UNUnbekannt

Senior Software Engineer - AI Interaction Evaluator(m/w/x)

Wien
from USD 50 - 200 / hour
Full-timeFreelanceRemoteSenior
AI/ML
Apply now

Evaluating AI-generated coding interactions for relevance and authenticity. Strong software engineering judgment and experience with TypeScript, Python, and OpenAI Codex required. Take-home evaluation exercise provided.

Requirements

  • Highly experienced software engineer (SR+)
  • Staff / Principal-level engineer or equivalent experience
  • Strong background in TypeScript / JavaScript
  • Strong background in Python
  • Hands-on experience with OpenAI Codex
  • Hands-on experience with Claude Code
  • Hands-on experience with Cursor
  • Deep familiarity with modern AI-assisted dev workflows
  • Able to evaluate code without full execution
  • Comfortable giving direct, opinionated feedback
  • High bar for good engineering
  • Experience with AI-first IDEs like Cursor
  • Prior exposure to prompt design or evaluation
  • Experience mentoring senior engineers
  • Experience defining engineering standards

Tasks

  • Evaluate AI-generated coding interactions
  • Assess response relevance and usefulness
  • Judge preamble and reasoning quality
  • Determine if output reflects strong engineering judgment
  • Evaluate interaction authenticity for experienced developers
  • Assess overall usefulness and correctness of outputs
  • Determine alignment with strong engineering practices
  • Evaluate quality of explanations and reasoning
  • Distinguish between different levels of response quality
  • Provide clear, opinionated feedback on what worked
  • Identify what didn’t work in AI responses
  • Highlight what felt off or misleading
  • Define standards for great interactions with tools like Cursor
  • Determine if responses feel authentic to a strong engineer
  • Assess if explanations are helpful beyond technical correctness
  • Evaluate if the model guides users effectively
  • Determine if interactions build or erode trust

Work Experience

  • approx. 4 - 6 years

Education

  • Bachelor's degreeOR
  • Master's degree

Languages

  • English – Business Fluent

Tools & Technologies

  • TypeScript
  • JavaScript
  • Python
  • OpenAI Codex
  • Claude Code
  • Cursor

Benefits

Other Benefits

  • Take-home evaluation exercise
  • One behavioral interview
Find the original job posting in its most current version here. Nejo automatically captured this job from the website of Unbekannt and processed the information on Nejo with the help of AI for you. Despite careful analysis, some information may be incomplete or inaccurate. Please always verify all details in the original posting! Content and copyrights of the original posting belong to the advertising company.

Like this job?

Beta

Your Career Agent finds similar jobs for you every day.

Not a perfect match?

100+ Similar Jobs in WienView all
  • Anyline

    Software Engineer(m/w/x)

    Full-timeWith HomeofficeExperienced
    Wien
    from 60,000 / year
  • Remove.tech

    AI Software Engineer(m/w/x)

    Full-timeWith HomeofficeExperienced
    München, Wien
    from 70,000 - 85,000 / year
  • Canva

    Senior Machine Learning Engineer - Multimodal Data(m/w/x)

    Full-timeWith HomeofficeSenior
    Wien
  • Flinn.ai

    Senior Fullstack Engineer(m/w/x)

    Full-timeWith HomeofficeSenior
    Wien, Berlin
  • Mimica

    Head of/VP of Engineering(m/w/x)

    Full-timeRemoteSenior
    Wien
View all 100+ similar jobs
Apply now