Provide senior-level legal expertise to design, evaluate, and benchmark AI systems for legal tasks. Review AI outputs, create task specifications and reference solutions, develop evaluation frameworks and scoring guidelines, and give structured feedback. Collaborate with researchers, engineers, and program teams to translate professional legal judgment into consistent evaluation standards.
This role is for one of our clients
Compensation: $85-$120 per hour
We are seeking an experienced counsel-level legal professional to work with a leading AI research organization’s research and program management teams to improve how advanced AI systems understand, reason about, and perform real-world legal work.
Your legal expertise will play a central role in defining what high-quality legal reasoning looks like. You will review legal knowledge-work tasks and AI-generated outputs, develop detailed instructions and benchmark solutions, and help establish evaluation standards that accurately reflect professional legal practice.
This is a full-time role requiring 40 hours per week for an initial 6-month engagement. The position follows a hybrid working model in the Bay Area, California, with regular on-site collaboration required.
RequirementsKey Responsibilities
- Legal Quality Review: Evaluate legal tasks and AI-generated responses for accuracy, completeness, sound reasoning, and practical applicability.
- Instruction & Benchmark Development: Create detailed task specifications, develop high-quality reference solutions, and design legal scenarios that reflect authentic professional workflows.
- Evaluation Frameworks: Develop and refine legal evaluation criteria, scoring guidelines, and benchmarks used to measure AI performance.
- Domain Expertise: Apply specialized knowledge to identify missing legal considerations, flawed reasoning, unsupported conclusions, and responses that may appear credible but fail professional scrutiny.
- Cross-Functional Collaboration: Work closely with researchers, program managers, engineers, and other legal specialists to improve evaluation methodologies.
- Calibration & Standards: Translate professional legal judgment into clear, consistent, and teachable evaluation standards.
- Written Feedback: Provide precise, structured feedback explaining why a legal response or workflow meets or falls short of professional expectations.
- Education: Juris Doctor (JD) from an accredited law school; education from a highly regarded institution is preferred.
- Experience: 8–15 years of substantive post-qualification legal experience within an established law firm, corporate legal department, regulatory organization, court, or comparable institution.
- Seniority: Currently operating at or having reached Counsel, Senior Associate, Senior In-House Counsel, or Assistant General Counsel level, with meaningful ownership of legal matters.
- Legal Specialization: Deep expertise in at least one substantive practice area, such as:
- Corporate & Transactional Law
- Litigation & Dispute Resolution
- Regulatory & Compliance
- Intellectual Property
- Employment & Labor Law
- Tax Law
- Licensure: Admission to at least one U.S. state bar and in good standing.
- AI Fluency: Practical familiarity with large language models and AI-assisted workflows, with the ability to distinguish genuinely sound legal reasoning from plausible but incorrect analysis.
- Communication: Exceptional written communication skills and the ability to articulate complex legal judgments clearly and systematically.
- Availability: Ability to commit reliably to 40 hours per week for an initial six-month engagement.
- Location: Currently based in the Bay Area, California, with the ability to work on-site multiple days per week as required. Candidates outside the Bay Area must be willing to relocate at their own expense; relocation assistance is not provided.
You will collaborate closely with research and program teams and may use organization-provided systems, accounts, and equipment. The role requires comfort working in a fast-paced environment where priorities and evaluation requirements may evolve as the AI research progresses.
Equal Employment OpportunityWe are committed to providing equal employment opportunities to all qualified applicants. Employment decisions are made without regard to legally protected characteristics, and reasonable accommodations are available to qualified individuals with disabilities and disabled veterans throughout the application process.
Engagement Details- Role: Senior Counsel — Legal Domain Expert
- Employment Type: Full-Time
- Commitment: 40 hours per week
- Initial Engagement: 6 months
- Work Model: Hybrid
- Location: Bay Area, California
- Work Arrangement: On-site collaboration required multiple days per week
- Relocation: Candidates outside the Bay Area must relocate at their own expense
- Focus: Legal AI training, evaluation, benchmarking, and domain expertise
Similar Jobs
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
Promote pharmaceutical products through academic visits and medical information delivery, build relationships with healthcare experts, understand customer needs, execute market strategies, influence clinical perceptions, increase product market share, and maintain a strong professional and company image. The role prioritizes candidates with oncology experience, particularly in lung cancer, breast cancer, urology, or medical oncology, as well as strong account management and teamwork skills.
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Develop and operate CrowdStrike’s internal cloud management platform and mission-critical CI/CD pipelines. Build APIs and UIs using Go and TypeScript, improve developer productivity, integrate tooling, maintain cloud-native infrastructure, and support secure, highly available software delivery systems. Collaborate with development, infrastructure, security, and SRE teams, ensure SOC2-aware processes, and participate in on-call operations.
Top Skills:
AlertmanagerArgo CdArgo WorkflowsCi/CdDebianDockerGoGoreleaserGrafanaHelmJavaScriptJenkinsKubernetesKustomizeLinuxPrometheusPulumiPypiTemporalTerraformTypescript
Machine Learning • Payments • Security • Software • Financial Services
Provides high-volume technical support through phone, ticketing, and chat channels. Troubleshoots hardware, software, network, access, and application issues; manages incidents in ServiceNow; handles escalations; and communicates solutions to nontechnical users. Guides junior analysts, supports knowledge management and training, analyzes trends, recommends process improvements, and contributes to IT service efficiency and compliance.
Top Skills:
ItilItsmServicenow
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
