Build and validate challenging Codebase Q&A tasks for AI agents using production Go repositories. Responsibilities include exploring unfamiliar codebases, understanding runtime behavior and history, rewriting engineering questions, refining rubrics and foils, running validation workflows, documenting tasks, and responding to expert review. Work is fully remote, asynchronous, and paid per approved task.
This role is for one of our clients
Compensation: $130 per task
We are seeking experienced Go engineers to build the dataset that teaches AI agents to reason about real codebases.
Codebase Q&A is a reinforcement-learning environment for training AI agents to explore and reason about software repositories they have never seen before. You will build difficult codebase-exploration tasks: high-level engineering questions about production Go repositories that cannot be answered by reading a single file, and that require runtime evidence to resolve — the kind of question a senior engineer answers by actually investigating the system.
This track works on infrastructure-grade Go codebases including etcd, Traefik, Helm, Kustomize, CoreDNS, NATS Server, Temporal, Syncthing, restic, Authelia, BadgerDB, quic-go, gRPC-Go, Hugo, rclone, Trivy, cert-manager and Pion WebRTC.
Contract, task-based work paid at $130 per approved task, performed remotely in our Studio.
Requirements
Key Responsibilities
- Select tasks from a pre-validated pool of Go repositories, each pinned to a specific git commit and arriving with an engineering question, positive and negative rubrics, foils, and a golden solution
- Explore the repository until you genuinely understand the subsystem in question, including the relevant PR or issue history
- Rewrite the task's question so it defeats frontier coding agents while remaining well-defined and fairly answerable
- Adjust rubrics and foils so they reward correct reasoning and reject plausible-but-wrong answers
- Run the full validation loop in Studio — Check, Golden, Run, Analyze — and iterate until every stage passes
- Document your work in the task README and submit for expert review, addressing reviewer feedback on returned tasks
Ideal Qualifications
- 3+ years of professional software engineering experience, with substantial production Go
- Demonstrated ability to navigate a large, unfamiliar Go codebase and explain how it behaves at runtime — not just what the source says
- Fluency in Go concurrency, interfaces, module boundaries, and the standard library; comfort reading generated code and build tooling
- Experience with distributed systems, networking, storage engines, or Kubernetes-ecosystem tooling is highly relevant — the task mix is weighted toward architecture and system design (42%), code onboarding (26%) and root-cause analysis (20%)
- Precise written English: the questions and rubrics you write are the product
- Comfortable with git at commit level, containerized environments, and command-line tooling
Working Details
- Fully remote and asynchronous, with expert team leads across US, India, and Nigeria time zones
- Work is performed in our Studio; time is tracked with Insightful, and platform access is provisioned through Okta
- Onboarding requires identity verification, a background check, signed Terms of Work and CIIAA, a tax form, and passing a short calibration quiz before task access is granted
- Payment is per approved task; every task is reviewed by a second engineer and a super reviewer before acceptance
Application Process
- Submit your resume or relevant technical background to get started
- Qualified applicants may be asked to complete a brief technical assessment or provide additional information
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Similar Jobs
Artificial Intelligence • Enterprise Web • Information Technology • Productivity • Sales • Software • Database
Own Apollo’s core data catalog product vision, strategy, roadmap, and delivery. Lead data quality, telemetry, sourcing, vendor evaluation, taxonomy, enrichment, and customer-facing data experiences. Partner with Engineering, Data Science, Design, GTM, Support, and Legal to improve catalog coverage, accuracy, freshness, compliance, and usability. Prototype rapidly with AI-native tools, define KPIs, make build-versus-buy decisions, and drive cross-functional execution for Apollo’s data platform.
Top Skills:
APIsCcpaClaude CodeCursorData ModelsData TelemetryFigma AiGdprKpi InstrumentationLovableObservability
Information Technology • Internet of Things • Mobile • On-Demand • Software
Sells Spectrum internet, TV, mobile, and voice products in a retail store while meeting sales and service goals. Builds customer relationships, educates customers on products and promotions, handles payments, resolves issues, maintains product and visual standards, minimizes losses, and supports a welcoming store environment. The role requires flexible scheduling, prolonged standing, lifting up to 35 pounds, strong communication, cash-handling ability, technology comfort, and a high school diploma or equivalent.
Top Skills:
Computer ApplicationsMobile DevicesTelecommunicationsVideo Platforms
Healthtech • Pet
Chief of Staff responsible for building and leading internal operations, improving company systems and processes, representing leadership, driving cross-functional initiatives, managing a small operations team, strengthening employee experience, and improving internal communications. The role requires strong judgment, discretion, initiative, organizational ability, communication skills, and experience operating effectively in a demanding, growing organization.
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
