Senior Python Software Engineer – Coding-Agent Safety Evaluation

About the role

We are looking for a senior software engineer to design and evaluate realistic coding‑agent benchmark tasks. The role blends production engineering, secure software development, repository validation, test design, and safety evaluation to ensure agents complete tasks without violating safety, privacy, or data‑integrity constraints.

Key responsibilities

• Build production‑like benchmark tasks using repositories, tests, configuration, documentation, and runtime scenarios.
• Create realistic engineering tasks such as bug fixes, feature additions, CI repairs, configuration migrations, integration updates, and runtime‑state fixes.
• Define utility, safety, and alignment requirements that protect system constraints, developer intent, permissions, privacy, oversight, and data integrity.
• Develop visible and hidden test suites and produce safe and unsafe reference solutions for controlled evaluation.
• Package prompts, metadata, runnable repositories, evaluators, reference patches, scoring rubrics, and calibration notes.
• Analyze coding‑agent rollouts for utility completion and safety violations.
• Collaborate with engineering, QA, security, and client teams to improve evaluator reliability and benchmark difficulty.

Required profile

• Bachelor's or Master's degree in Computer Science or a related technical discipline.
• At least 8 years of hands‑on software engineering experience in tier‑one product companies or technology startups.
• Strong professional experience with Python and at least one of Java, Go, or C++.
• Proven experience building and maintaining production software used by real customers.
• Deep knowledge of software architecture, debugging, performance optimization, and production engineering (deployment pipelines, monitoring, logging, observability, metrics, distributed tracing, high availability, fault tolerance, scalability, disaster recovery).
• Secure‑software experience identifying and fixing vulnerabilities (injection, authentication/authorization, memory safety, race conditions, deserialization, secrets management, input validation).
• Practical experience with incident response and building resilient systems.

Required skills

• Python
• Java
• Go
• C++
• Production engineering
• Deployment pipelines
• Monitoring and logging
• Observability and metrics
• Distributed tracing
• High availability and fault tolerance
• Scalability and disaster recovery
• Secure software development (injection, auth, memory safety, race conditions, deserialization, secrets management, input validation)

What we offer

• Remote contractor role (8 hours/day with 6 hours overlap with Pacific Time).
• Engagement of 4–8 weeks, starting around 7 September 2026.
• Compensation of $28 per hour.