Senior Software Engineer, Ailevate

Ankura is a team of excellence founded on innovation and growth.

Practice Overview:

AI is more than just chatbots—it is the foundation for autonomous intelligence, decision-making, and scalable automation. At Ailevate, an Ankura company, we are pioneering the next evolution of Agentic AI, where autonomous AI Agents leverage Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and real-time data processing to drive impact across industries.

Our platform is built for businesses that need scalable, privacy-first AI solutions to enhance efficiency, automation, and decision-making. Join us in shaping the future of AI beyond chatbots, where LLM-powered Agents solve complex problems, streamline workflows, and optimize analytics-driven insights.

Role Overview:

The ideal candidate selected for this role will ideally be working US eastern time hours. As a Senior Software Engineer (Python), you will be a core contributor to our Agentic AI Platform, designing and scaling microservices that power LLM-driven AI Agents. You will collaborate with AI researchers, data engineers, and cloud infrastructure teams to deploy AI models, optimize inference, and eventually fine-tune LLMs for industry-specific applications.  Your work will directly impact how AI-driven agents extract insights, process structured and unstructured data, and enhance real-time analytics.

This is a great opportunity to work on cutting-edge AI applications beyond chatbots, developing LLM-powered AI Agents that solve real industry problems.  This is a fully remote position with a highly talented AI-first team. The selected candidate will have ownership and impact to help shape a next-generation AI platform with real-world applications. Ankura offers competitive compensation and a strong work culture that values both innovation and work-life balance. Based on the selected candidates’ experience and qualifications, this role will be filled at either the Senior Associate or Director level within Ankura’s structure.

Responsibilities:

• Develop, optimize, and scale backend services using Python and FastAPI.

• Design and implement microservices for LLM-powered AI Agents, focusing on real-time processing, inference, and decision-making.

• Integrate LLM APIs (OpenAI, Anthropic, vLLM, etc.) to power AI-driven insights and automation.

• Enhance our Retrieval-Augmented Generation (RAG) pipeline, enabling AI Agents to retrieve, process, and synthesize knowledge.

• Implement messaging and event-driven workflows using RabbitMQ.

• Fine-tune and optimize LLMs using TensorFlow and PyTorch as the platform evolves.

• Deploy and manage AI workloads on Kubernetes, ensuring scalability and high availability.

• Collaborate with infrastructure and DevOps teams to streamline CI/CD pipelines and cloud-based deployments.

• Write well-structured, maintainable, and testable code following best practices.

• Mentor junior engineers and contribute to technical decision-making.

Requirements:

• 5+ years of Python software development experience.

• Strong experience with FastAPI (or Flask) for building scalable APIs.

• Experience with LLMs, NLP, and AI-driven applications.

• Experience integrating LLM APIs such as OpenAI, Anthropic, or vLLM.

• Proficiency in microservices architecture and distributed systems.

• Familiarity with frameworks such as TensorFlow and PyTorch and model optimization techniques.

• Experience with SQL and NoSQL databases such as Elasticsearch or PostgreSQL.

• Cloud experience with Azure, AWS, or GCP, along with CI/CD automation.

• Proficiency in containerization with Docker and orchestration using Kubernetes.

• Experience with event-driven architectures using RabbitMQ or similar message brokers.

• Strong problem-solving and debugging skills with a focus on performance optimization.

• Preferred to have experience fine-tuning LLMs and optimizing inference workloads.

• Preferred to have familiarity with Neo4j, graph databases, or knowledge graphs.

• Preferred to have experience contributing to open-source AI or LLM projects.

• Preferred to have understanding of vector databases such as Elasticsearch, Weaviate, or Pinecone.

• Preferred to have experience in healthcare, fintech, or other highly regulated industries.

*

Ankura is an Affirmative Action and Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against based on disability. Equal Employment Opportunity Posters, if you have a disability and believe you need a reasonable accommodation to search for a job opening, submit an online application, or participate in an interview/assessment, please email accommodations@ankura.com or call toll-free +1.312-583-2122. This email and phone number are created exclusively to assist disabled job seekers whose disability prevents them from being able to apply online. Only messages left for this purpose will be returned. Messages left for other purposes, such as following up on an application or technical issues unrelated to a disability, will not receive a response.