About the job

Mistral AI is hiring an AI Engineer to join the Le Chat product team, where you will be responsible for building the evaluation and A/B testing framework, adding observability, and managing reliable model releases to improve the performance and safety of LLM systems. You will define and track key metrics such as task success, helpfulness, hallucination proxies, latency, and cost, run A/B tests on prompts and models, set up observability with structured logging and dashboards, and operate model release processes with canary/shadow traffic and rollback criteria. You will also improve system behaviors including memory policies, intent classification, and tool-call reliability, while creating templates and documentation to empower other teams. The ideal candidate has strong Python or TypeScript skills, hands-on production LLM experience, and expertise in evals, A/B testing, observability, and product-oriented experimentation. Additional desirable skills include safety systems (moderation, PII redaction), release operations, and familiarity with automated rollbacks. The hiring process includes introductory calls, technical interviews (ML and algorithm coding), and a culture-fit discussion. The role is based in Paris with relocation support, but remote options are available across France, UK, Germany, Netherlands, Spain, and Italy with onboarding in Paris and regular monthly visits. Mistral offers a competitive salary, equity, health insurance, pension plan, transport and sports allowance, meal vouchers, generous parental leave, and visa sponsorship.