DistantJob Logo

DistantJob

Applied AI Engineer

Posted 7 Days Ago
Remote
Hiring Remotely in CAN
Entry level
Remote
Hiring Remotely in CAN
Entry level
Designs, builds, tests, and maintains AI conversations and agentic workflows for learner-facing products. Responsibilities include prompt and context engineering, structured outputs, RAG, tool integrations, state management, evaluation, observability, security, guardrails, and compliance controls. The engineer collaborates with Product, Engineering, Content Studios, and subject-matter experts to prototype, validate, deploy, and monitor reliable AI behavior.
The summary above was generated by AI

Teaching AI to say yes is easy. Teaching it when to say ‘no,’ ‘I don’t know,’ or ‘let me get a human’ is the actual job. Our client is hiring an Applied AI Engineer to build the second kind.



The Applied AI Engineer designs, builds and tests the AI conversations and agents inside the client’s learner-facing products: advisory experiences, practice simulations and diagnostic coaching.

They work with Product, Content Studios, Engineering and subject-matter experts. They turn product and instructional requirements into reliable AI behavior, prove it through testing, and keep it performing after release.

This company is at the forefront of learning technologies, providing e-learning programs for professionals in regulated industries such as healthcare and finance. They’re constantly evolving and growing, run efficient Agile-first teams, and have a broadly distributed staff. They value good-natured, inquisitive, and methodical IT professionals who come prepared to collaborate and share their knowledge.


Your responsibilities will include:

  • Turn scenario specs, decision models and assessment rubrics into prompts, application logic and testable behavior.
  • Spot missing requirements and technical constraints, and raise them with Product, Engineering and SMEs.
  • Define how AI models connect to application services and approved data sources: APIs, tool interactions, data structures, validation, error handling, permitted actions.
  • Design prompts, structured outputs and validation logic, and catch invalid responses before they reach learners.
  • Build conversational runtimes with controlled personas, session state and explicit state transitions.
  • Keep simulation behavior, assessment logic and feedback separate, so feedback cannot change assessment results.
  • Implement grounding and retrieval from approved knowledge sources, with source references where required.
  • Keep code, prompts, schemas and evaluation assets under version control, and document them.
  • Run bounded experiments with Product, with clear questions and success measures.
  • Take validated prototypes into production with Engineering, and check behavior stays the same.
  • Help build the technical controls that keep AI features compliant with accreditation requirements (issue flagging, review steps, disclosure, logging).

What are the required skills?

  • Expert prompt and context engineering: system prompts, few-shot examples, prompt chaining, routing, breaking complex tasks into steps
  • Structured outputs with JSON Schema, function calling and validation layers, with retry and fallback logic
  • RAG design: chunking, metadata filtering, hybrid search, reranking, citation grounding
  • Managing token budgets, prompt caching, context compaction and long-session memory
  • Agentic systems: tool use, planner/executor patterns, multi-agent orchestration, state machines, human-in-the-loop checkpoints
  • Defining tool schemas, permission scopes and action limits for agents
  • AI evaluation: golden datasets, rubric-based scoring, LLM-as-judge calibrated against human raters, regression testing
  • LLM observability and tracing
  • LLM security: prompt injection, data leakage, jailbreaks, unsafe tool actions, and guardrails
  • Comparing prompting, retrieval and model options on quality, latency and cost
  • Python or TypeScript, including API integrations and automated testing
  • Git (e.g. Bitbucket), reproducible tests and CI
  • Works with AI coding agents by default
  • Explains technical trade-offs to non-technical colleagues
  • Sound judgment on privacy, human oversight and the limits of AI-generated assessment or advice
  • Hands-on experience building LLM applications with multi-step conversational workflows
  • Experience collaborating with subject-matter experts

Preferred
  • Background in learning products, assessment, simulations or regulated professional content



If this looks interesting, please use the form below to apply! Remember, this is a fully remote, long-term opportunity :)

Similar Jobs

3 Days Ago
In-Office or Remote
180K-300K Annually
Junior
180K-300K Annually
Junior
Artificial Intelligence • Information Technology • Software
Build and deploy production voice AI applications for Spanish-speaking enterprise customers. Integrate APIs and SDKs, develop conversational workflows, debug issues across telephony and infrastructure, optimize prompts and AI reliability, and guide customers from proof of concept through launch. Lead technical onboarding, advise on architecture and deployment, collaborate with Product and Engineering, and turn customer feedback into scalable platform improvements.
Top Skills: APIsAWSAzureDistributed SystemsGCPLlmsPythonReal-Time CommunicationsSdksSipSpeech TechnologiesTelephonyTwilioTypescriptVoice Ai
17 Days Ago
Remote
150K-180K Annually
Senior level
150K-180K Annually
Senior level
Cloud • Information Technology • Software
Lead applied AI initiatives from problem scoping and experimentation through production deployment. Build and fine-tune LLM, agentic AI, computer vision, NLP, and multimodal systems; establish MLOps workflows; evaluate model performance; and make trade-offs across accuracy, interpretability, latency, and cost. Partner with product and engineering teams, communicate technical decisions to stakeholders, and contribute to AI research rigor and technical direction.
Top Skills: Agentic AiAWSAzureCi/CdComputer VisionContainerizationDiffusion ModelsGCPGitImage ProcessingJaxLarge Language ModelsMlopsModel DistillationMultimodal LearningNlpPythonPyTorchTensorFlow
18 Days Ago
In-Office or Remote
Senior level
Senior level
Fintech • Payments • Software • Financial Services
Build and scale production AI agents that automate financial workflows. Responsibilities include developing the shared agent harness, context and tool management, evaluation, monitoring, observability, permissions, approvals, audit trails, and customer-facing interfaces. The role partners with product teams, ensures agents act safely with financial data and transactions, tracks advances in models and inference, and establishes technical patterns and quality standards across the agent platform.
Top Skills: Agent InfrastructureAi AgentsContext ManagementEvaluation FrameworksFrontier ModelsGenerative UiHuman-In-The-Loop LabelingLlm-As-JudgeLlm-Backed SystemsLlmsModel InferenceMonitoringObservability

What you need to know about the Boston Tech Scene

Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.

Key Facts About Boston Tech

  • Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
  • Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
  • Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
  • Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account