Apply for the Competitive Programming — Test Engineer/Senior Reviewer
We are looking for a Senior Reviewer / Competitive Programming Engineer to help build and review high-quality algorithmic problems and automated graders used to train advanced AI coding models. You’ll have a very similar profile to a Problem Author, including creating problem packages when needed, while also reviewing and QAing the work produced by other experts. This includes verifying correctness, identifying edge cases, validating constraints, and ensuring the overall quality of the datasets. This role is ideal for competitive programmers with strong algorithmic problem-solving skills and deeper experience, particularly IOI/ICPC alumni and candidates with a Codeforces rating of 2400+ or IOI/ICPC medalist.
Key Responsibilities
- Review and QA competitive-programming content produced by other experts.
- Verify the correctness of problems, solutions, and test cases.
- Identify edge cases and potential issues in problem packages.
- Validate problem constraints and ensure they are appropriate for the intended problem.
- Review test cases to ensure they adequately cover potential incorrect solutions.
- Create problem packages when needed, including problems, solutions, incorrect solutions, and test cases.
- Develop and review comprehensive test cases, including edge cases, stress tests, generators, and adversarial/"hacking" tests.
- Review automated graders and checkers to ensure they accurately evaluate submitted solutions.
- Identify subtle issues in solutions and test cases.
- Work with other problem authors and engineers to maintain a high standard of correctness and quality.
- Help ensure problems and test suites provide reliable training signals for advanced AI coding models.
Core Requirements
- Senior-level competitive programming ability, ideally equivalent to a Codeforces rating (2200–2600) or demonstrated equivalent performance.
- Strong knowledge of algorithms, data structures, complexity analysis, and problem-solving techniques.
- Expert proficiency in C++.
- Strong proficiency in Python.
- Experience creating or solving challenging competitive-programming problems.
- Strong understanding of test generation, edge cases, stress testing, and adversarial testing.
- Ability to identify subtle bugs and construct test cases that reliably expose incorrect solutions.
- Excellent written and verbal communication skills in English.
- Ability to work independently in a remote, fast-paced environment.
Nice-to-Have
- IOI or ICPC participation, particularly at a high level.
- Competitive programming experience on Codeforces, AtCoder, CodeChef, or similar platforms.
- Experience writing programming contest problems, validators, checkers, or test generators.
- Experience working on coding benchmarks, automated evaluation systems, or AI/ML coding datasets.
- Experience developing deliberately-wrong solutions to model common programming mistakes.
Additional
- Project Length: 4–5 months initially, with potential for extension.
- Timezone: US Time zones.
- Seniority: Senior-level competitive programming. Codeforces rating of 2400+ or IOI/ICPC medalist.
- Particularly encouraged to apply: IOI/ICPC alumni and top competitive programmers.