Senior Azure GenAI Quality Engineer

Seargin is a dynamic multinational tech company operating in 50 countries.

At Seargin, we drive innovation and create projects that shape the future and greatly enhance the quality of life. You will find our solutions in the space industry, supporting scientists in the development of cancer drugs, and implementing innovative technological solutions for industrial clients worldwide. These are just some of the areas in which we operate.

Currently, for the new Seargin project, we are looking foran experienced Azure GenAI Quality Engineer.

ABOUT THE OFFER

Model: Fully remote

Start: ASAP is possible

Language: English (C1)

Recruitment process: 1 interview

Contract: B2B/Permanent

FTE: 1 (full-time)

ABOUT THE POSITION

We are seeking a senior-level engineer with deep expertise in Quality Assurance, Python development, Azure Cloud, and Generative AI systems. This includes hands-on experience designing and testing RAG architectures, deploying and evaluating LLM models in Azure AI Studio or Azure OpenAI, and building advanced automated testing frameworks in Python. The ideal candidate brings extensive experience in QA automation, strong knowledge of LLM evaluation methods, and solid proficiency in Azure DevOps, CI/CD, and DataOps workflows.

We value candidates who have worked with agentic AI frameworks, advanced Azure AI Search features, and modern orchestration tools such as LangChain or Semantic Kernel for building testing pipelines. Experience creating evaluation workflows with Prompt Flow, developing ground-truth datasets, or holding an advanced academic background in AI or Data Science will significantly strengthen your profile.

Additional strengths include hands-on experience fine-tuning Small Language Models (SLMs), familiarity with Azure ML environments, or supplementary coding skills in C# / .NET. Knowledge of red-teaming techniques, experiment tracking tools like MLflow, or certifications such as Azure AI Engineer Associate (AI-102) will be considered valuable bonuses.

QUALIFICATIONS

Non-negotiable skills

Generative AI

• 1–2 years of Generative AI experience.

• Practical hands-on experience with RAG architecture (Vector Search + LLM).

Azure Cloud

• Experience with Azure AI Studio (or Azure OpenAI Service) — must have deployed/tested models in Azure, not just locally.

Coding (Python)

• Python Specialist: Able to write complex scripts (Pandas, PyTest), not basic Selenium-type scripting.

QA & Automation

• Deep understanding of LLM evaluation metrics, including “LLM-as-a-Judge” vs deterministic assertions.

DataOps / CI/CD

• Azure DevOps pipelines that automatically trigger AI evaluations on pull requests.

Preferred skills

Generative AI

• Experience with Agentic AI (LangGraph, AutoGen) or multi-agent systems.

Azure Cloud

• Experience with Azure AI Search (Cognitive Search), including deep knowledge of indexing, vector profiles, and hybrid search.

Coding (Python)

• Experience with LangChain or Semantic Kernel for building test harnesses.

QA & Automation

• Experience using Prompt Flow to build evaluation pipelines.

DataOps / CI/CD

• Experience creating Golden Datasets (ground truth) for regression testing.

Education

• Master’s degree in Data Science or an AI-related field.

Nice to have:

Generative AI

• Experience fine-tuning Small Language Models (SLMs) like Phi-3 or Llama.

Azure Cloud

• Experience with Azure ML (AML), Workspaces, and Compute Instance management.

Coding (Python)

• Background in C# / .NET to integrate with the client’s backend.

QA & Automation

• Experience with Red Teaming (security, jailbreak, adversarial testing).

DataOps / CI/CD

• Experience with MLflow for experiment tracking.

Education

• Certification: Azure AI Engineer Associate (AI-102).

Discover the Power of IT Excellence. Apply!

To learn more about Seargin, please visit our web page: www.seargin.com