Senior Python Engineer – AI Coding Agent Evaluation (Freelance)

About the role

Mindrift is seeking a senior Python engineer to help build a high‑quality dataset for evaluating AI coding agents. You will design realistic developer environments and craft challenging tasks that test the limits of current models.

Key responsibilities

• Create virtual company environments with codebases, infrastructure, tickets, documentation, and conversation histories.
• Design intermediate‑state tasks, write clear prompts, and define objective success criteria.
• Develop functional and integration tests that accept all correct solutions while rejecting incorrect ones.
• Iterate on tasks and tests based on QA feedback, analyzing agent failures and refining evaluation fairness.

Required profile

• 8+ years of software development experience.
• Strong command of English (minimum B2 level).
• Experience writing robust functional and integration tests.

Required skills

• Python (FastAPI)
• JavaScript / TypeScript (React)
• Docker
• PostgreSQL
• Kafka
• Redis

What we offer

• Freelance contract with flexible, remote work.
• Compensation up to $150 per hour, based on expertise and pace.
• Project‑based engagement allowing you to set your own schedule.