Build and operate an on-prem AI platform (Model-as-a-Service) including RAG pipelines, vector DBs, MCP tooling, agent repositories, CI/CD integration, and guardrails within MRM. Optimize large on-prem GPU infrastructure and translate business use cases into scalable platform services.
Role Overview
We are looking for an AI Platform Engineer—a builder who can architect the "factory" where AI is made.
Our goal is to build an internal, on-premises AI ecosystem that mimics the capabilities of AWS or Azure. You will be responsible for creating a horizontal platform used by various lines of business to deploy AI projects simultaneously.
Key Responsibilities
- Platform Architecture: Design and develop a "Model-as-a-Service" platform that allows non-experts to use drag-and-drop components to build AI solutions.
- RAG-as-a-Service: Build and optimize end-to-end Retrieval-Augmented Generation (RAG) pipelines, including sophisticated chunking strategies and vector database management.
- Tooling & Libraries: Develop and maintain MCP (Model Control Protocol) libraries, clients, and servers to connect various data sources to the AI engine.
- Infrastructure Management: Help manage and optimize one of the largest on-premise GPU farms in the U.S. banking sector (500+ Nvidia nodes).
- Agentic AI: Build a repository for Agentic AI where users can select existing agents or build custom ones for specialized tasks.
- CI/CD Integration: Integrate AI deployment pipelines with enterprise-level CI/CD tools like Jenkins and Ansible.
- Compliance & Guardrails: Implement corporate-level guardrails and work within Model Risk Management (MRM) frameworks to ensure all AI deployments are secure and compliant.
Required Technical Skills
- Expert Python: Deep, hands-on knowledge is mandatory.
- Data Engineering: Extensive experience in massive data ingestion and processing.
- RAG Expertise: Deep understanding of vector databases, inferencing, and advanced chunking strategies.
- Platform Engineering: Proven experience building tools/platforms that other developers or business units use.
- Infrastructure Knowledge: Experience mimicking cloud capabilities (AWS/Azure) within a strictly on-premise environment.
- DevOps: Familiarity with Jenkins, Ansible, and automated deployment pipelines.
Experience & Qualifications
- Seniority: This is a senior-level role. We are looking for someone with a proven track record of building production-grade platforms (10-15+ years)
- Industry Knowledge: You must stay current with the "latest and greatest" in AI (e.g., rag-less inferencing, agentic frameworks).
- Problem Solver: Must be able to take a use case from a business unit and translate it into a scalable platform service.
- Experience with Scale: Experience working with large-scale GPU farms and high-volume data environments is highly preferred.
Compensation, Benefits and Duration
Minimum Compensation: USD 48,000
Maximum Compensation: USD 168,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and a good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full time employees.
This position is available for independent contractors
No applications will be considered if received more than 120 days after the date of this post
Similar Jobs
Cloud • Information Technology • Security • Software • Cybersecurity
Leads strategic relationships with private equity firms, developing joint go-to-market plans, originating portfolio company opportunities, structuring commercial deals, and driving adoption of Zscaler’s security platform. The role partners with PE leaders, sales teams, business value specialists, and technical experts to quantify value, improve EBITDA, mitigate cyber risk, support M&A transactions, and advance digital transformation across portfolio companies.
Top Skills:
Ai/MlCybersecurityDigital TransformationEbitdaIrrNetwork SecurityQbrsRoiSales ForecastingSaseTcoZero Trust Exchange
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Drive net-new business growth within a strategic enterprise technology account. Identify expansion opportunities, build business cases and solution roadmaps, develop relationships from IT teams through C-suite executives, coordinate cross-functional sales and partner teams, and close new solution areas while achieving license, renewal, professional services, and annual sales targets.
Top Skills:
Artificial IntelligenceCustomer Service ManagementHr Service DeliveryIt Operations ManagementIt Service ManagementSecurity OperationsServicenow
Artificial Intelligence • Fintech • Insurance • Marketing Tech • Software • Analytics
Handles inbound calls and warm leads to understand customers’ insurance needs, recommend appropriate Property & Casualty coverage, and convert prospects into policyholders. The role includes paid licensing and training, customer consultation, sales closing, brand representation, and maintaining a reliable remote work environment. Employees must work four weekdays and one weekend day, meet home-office and internet requirements, and remain in their resident state for at least one year.
Top Skills:
PcWired High-Speed Internet
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
