Backend Engineer (Python/JavaScript) – Remote, $15/hr

About the role

We are looking for a Backend Engineer with strong Python and JavaScript expertise to create multi‑agent benchmark tasks for AI coding agents. The position is a short‑term, remote contract (4 weeks) paying $15 per hour, with a commitment of 20‑40 hours per week and a required 4‑hour overlap with Pacific Standard Time.

Key responsibilities

• Design and build benchmark tasks based on real‑world open‑source code changes such as bug fixes, migrations, and refactors.
• Integrate tasks with the Harbor evaluation framework and run them inside Docker containers.
• Write clear task instructions, including file paths, function signatures, expected behavior, and constraints.
• Develop Python verification scripts to check the correctness of agent‑generated code.
• Create decomposition strategies to split complex changes across independent sub‑agents.
• Debug, refine, and ensure reproducibility of tasks in containerized environments.
• Analyze performance signals and continuously improve task quality, clarity, and difficulty.

Required profile

• Several years of professional experience developing in Python and JavaScript.
• Familiarity with AI coding benchmark suites such as SWE‑bench or Terminal‑Bench.
• Proven ability to navigate large open‑source codebases (e.g., Django, Flask, FastAPI, Node.js).
• Comfortable with Git workflows, including pull requests, diffs, and cherry‑picking.
• Experience working independently in a remote setting.

Required skills

• Python
• JavaScript
• Docker (Dockerfiles, image building, debugging)
• Git
• pytest or unittest for test script creation