Senior Software Engineer for Machine Learning Infrastructure

Company Intro

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains-from doctors and lawyers to physicists and engineers-and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, Poolside, Recraft, and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Israel, Czech Republic, Serbia, and more. We are headquartered in Amsterdam.

About the Role

Toloka AI’s Machine Learning, without exaggeration, a key part of Toloka’s success. ML Platform can be represented as a suite of tools and services designed to support the entire machine learning lifecycle, from data preprocessing to model deployment, management, and serving.

We are looking for a Senior Software Engineer who can help us make our platform even better. We expect the candidate to have a startup mindset – move fast, ready for pivoting, while building a reliable infrastructure.

What You’ll Work On

• Develop ML infrastructure (backend for LLM inference, compute and code execute services);
• Implement new tools for our internal clients;
• Maintain the platform;
• Develop technical documentation;
• Gather requirements from our internal ML teams and offer optimal solution balancing different use cases;
• Be the part highly skilled MLOps engineers;
• Collaborate with users of LLM models - Solution Engineers, ML Engineers to provide them with the best ml services and tools.

Requirements

• 5+ years experience as Software Engineer (primarily Python);
• Experience in infrastructure or backend development;
• Experience with C/C++: memory management and debugging, understanding of build systems (CMake/Make), ability to work with low-level code;
• Interest in working on internal infrastructure and MLOps;
• Ability to design solutions and maintain ML infrastructure;
• English language - B2 or higher.

Requirements +

• Understanding of instruments and practices for LLM training and inferencing;
• Experience with ML infrastructure;
• Familiar with popular DevOps tools: Kubernetes, Docker, Terraform;
• Knowledge of how to work with Spark (Databricks);
• Experience in working with Airflow (or there's ETL orchestration tools) and DBT;
• Experience integrating Python services with C/C++ modules is a plus;
• Familiarity with Java or other JVM languages is a plus.

Tools and technologies

• MLFlow, HuggingFace, vLLM, Celery, Databricks, Airflow, dbt, Microsoft Azure, Airbyte, k8s, terraform and a lot of custom self-developed tools.

What we can offer

• Freelance collaboration via the Mindrift platform (part of Toloka AI);
• Flexible, fully remote work schedule (20–30 hours per week), with the option to work on weekends;
• Hourly rate: from $30 to $60 (final amount to be determined after the final interview with the team);
• You’ll have the opportunity to work on highly innovative projects at the leading edge of AI development, together with a genuinely dedicated and dynamic team of experts;
• You’ll also be working on projects with customers that are AI industry leaders and well-known household names;
• Friendly community.