Optimize large-model inference at scale: improve throughput, reduce latency and cost per token, build benchmarking harnesses, tune parallelism and quantization strategies, implement load-balancing in routing, and evaluate custom kernels and emerging inference hardware.
Dragonfly is a crypto-native Venture Capital and research firm with $3.6B+ in assets under management and 160+ portfolio companies. Our Talent team connects people with roles across our portfolio, opening the door to opportunities through our Talent Network.
This is an application to join our talent network. This is not a listing for an internal role at Dragonfly.
We're actively sourcing for a Senior Inference Optimization Engineer for one of our portfolio companies building privacy-first consumer AI infrastructure. You'll be on the bleeding edge of LLM inference performance, pushing throughput, driving down latency, and optimizing cost per token at significant scale.
Location: Remote, USA (open to excellent candidates outside the USA)
What We’re Looking For:
- 5+ years in performance optimization or HPC with deep GPU architecture and parallel programming knowledge
- Hands-on experience with at least one production LLM inference engine (vLLM, SGLang) running at high volume
- Demonstrated experience with LLM inference optimization: continuous batching, PagedAttention, KV cache management, speculative decoding, quantization, CUDA graphs, torch.compile
- Experience with distributed inference strategies: tensor parallelism, pipeline parallelism, MoE parallelism in multi-GPU and multi-node environments
- GPU profiling fluency: Nsight Systems, Nsight Compute, PyTorch Profiler
- Proficiency in Python, Rust, or Go. C++/CUDA a strong plus
- Bonus: custom Triton kernels, diffusion/image model inference optimization, open-source inference framework contributions
About the role:
- Stand up and optimize GPU infrastructure including B300 nodes in owned data centers
- Drive down TTFT and TPOT, push throughput, and improve cost per token for LLM inference workloads
- Build reproducible benchmarking harnesses across inference engines to identify optimal engine, quantization scheme, and parallelism strategy per workload and GPU SKU
- Optimize multivariate inference load-balancing algorithms within the inference routing system
- Evaluate emerging inference optimization techniques including custom CUDA/Triton kernels, novel attention variants, new quantization schemes, and compilation stack improvements
- Evaluate emerging inference hardware (FPGAs, ASICs, custom silicon) for viability in the stack
Even if you don't match every point above but are an engineer passionate about AI and/or crypto, we encourage you to apply. There may be other opportunities that fit your skill set.
Process:
- We'll review your application and assess fit for this role.
- If there's a match, we'll facilitate a warm introduction to the team.
- If the timing isn't right, we'll keep you in mind for future opportunities across the portfolio.
Compensation
- Early Career: $110,000–$150,000
- Mid: $150,000–$200,000
- Senior: $180,000–$250,000
- Staff/Leadership: $230,000–$330,000
The compensation range reflects variation by seniority, location, and hiring company. Final compensation is confirmed with the specific hiring company.
Submit your information below, and we’ll reach out if there’s a potential fit.
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads measurement and product marketing for eight enterprise AI platforms. Builds shared KPI, taxonomy, scorecard, data-quality, and value-realization frameworks; translates usage and outcome data into executive insights and investment guidance. Oversees positioning, internal launches, campaigns, enablement, adoption, and audience segmentation. Partners across product, engineering, data, communications, and business teams while building and managing teams responsible for analytics, marketing, and communications.
Top Skills:
Ai PlatformsBusiness IntelligenceDashboardsData ContractsData FabricData VisualizationEvent TaxonomyExperimentationKnowledge GraphsKpi FrameworksOkr PlatformsProduct AnalyticsTelemetry
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads product strategy, roadmap, lifecycle ownership, adoption, and measurable outcomes for a greenfield enterprise AI platform. Defines platform boundaries, governance controls, evaluation, registration, and cost-tracking capabilities while translating evolving risk, privacy, security, and GxP requirements into usable product features. Partners with engineering, design, legal, compliance, risk, security, and global agent-building teams to prioritize investments, guide delivery, communicate direction, and drive platform adoption.
Top Skills:
AgileAIAi AgentsAi GovernanceGxpLeanModel Evaluation
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Build, test, and ship production-grade full-stack software and AI-powered systems for Pfizer’s Medical Affairs portfolio. Integrate LLMs, agentic AI, and RAG pipelines; develop scalable systems from prototypes through production; optimize prompts for cost, latency, and quality; integrate enterprise platforms; and monitor AI/ML performance. Collaborate with architecture, product, UX, and data engineering teams while maintaining strong code quality, documentation, testing, and verification practices.
Top Skills:
Agentic AiAgileApi-First DesignAWSAzureCi/CdGdprGxpHipaaLlmopsLlmsMlopsRagSalesforce Life Sciences CloudSalesforce Marketing CloudScrumVector DatabasesVeeva Crm
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
