Data Scientist -Data Scientist Responsible AI
Rabobank

Opdrachtomschrijving
Voor Rabobank in Utrecht werk je binnen het Responsible GenAI-team aan evaluatiemogelijkheden en guardrails voor generatieve AI. Je helpt teams binnen de organisatie om de kwaliteit en veiligheid van GenAI-oplossingen te begrijpen, te meten en te verbeteren. De toepassingen variëren van assistenten en rapportgeneratie tot documentverwerking en agentic applicaties.
Je ontwikkelt herbruikbare frameworks en hulpmiddelen voor herhaalbare tests, vergelijkingen en regressieanalyses. Daarbij ontwerp je statistisch onderbouwde experimenten, datasets en evaluatiescenario’s en valideer je metrics aan de hand van menselijke beoordelingen. Ook ontwikkel en beoordeel je guardrails voor onder meer blootstelling van gevoelige gegevens, niet-onderbouwde output, prompt injection en onbedoeld systeemgedrag.
De opdracht vraagt om een combinatie van data science, softwareontwikkeling en Responsible AI. Je vertaalt experimenten naar onderhoudbare, geteste en productieklare Python-code en werkt samen met engineers aan integratie in API’s, pipelines, CI/CD, monitoring en productieprocessen. Het resultaat bestaat uit betrouwbare evaluaties en praktische beheersmaatregelen waarmee meerdere teams hun GenAI-oplossingen verantwoord kunnen ontwikkelen en inzetten. De opdracht duurt zes maanden, met mogelijkheid tot verlenging, en start zo snel mogelijk. Hybride werken is van toepassing.
Gevraagd profiel
Rabobank zoekt een ervaren Data Scientist met praktische kennis van LLM-gebaseerde toepassingen, waaronder retrieval-augmented generation, prompting, grounding en contextmanagement. Je combineert statistische scherpte met een hands-on ontwikkelmentaliteit. Je kunt experimenten en prototypes omzetten in productieklare code en hebt ervaring met geautomatiseerde data- en evaluatiepijplijnen, versiebeheer, testen en reproduceerbare experimenten.
Je bent analytisch en kritisch: je beoordeelt of metrics daadwerkelijk meten wat ze moeten meten, onderzoekt onzekerheid en representativiteit en maakt duidelijk welke conclusies de data wel en niet ondersteunt. Daarnaast kun je guardrails selecteren, ontwikkelen en valideren en bevindingen helder uitleggen aan engineers, productmanagers, domeinexperts en andere stakeholders. Je werkt in het Engels.
Eisen
- The candidate must have practical experience with LLM-based applications and understand their limitations and failure modes
- The candidate must be familiar with retrieval-augmented generation, prompting, grounding and context management
- The candidate must be able to assess GenAI outputs against explicit criteria rather than relying only on intuition
- The candidate must understand how changes to models, prompts, data or application logic can affect quality and safety
- The candidate must have strong Python skills and demonstrated experience turning experiments and prototypes into maintainable, tested, production-ready code, not only exploratory notebooks
- The candidate must have practical experience integrating data-science components with APIs, structured and unstructured data, and automated data-processing or evaluation pipelines
- The candidate must have hands-on experience with version control, automated testing, CI/CD and reproducible experimentation
- The candidate must have sufficient production experience to understand deployment, monitoring and production workflows, diagnose issues and deliver code that engineers can integrate and operate
- The candidate must have demonstrated experience designing sound evaluations and experiments, and critically analysing metrics and datasets, not just applying evaluation frameworks
- The candidate must have strong applied statistical knowledge of sampling, dataset representativeness, uncertainty, metric reliability and experimental design, including the limitations of aggregate scores
- The candidate must have demonstrated experience validating automated evaluations against human assessments, assessing reliability and investigating disagreement
- The candidate must be able to distinguish between weaknesses in the GenAI solution, the evaluation dataset and the evaluator itself, and explain which conclusions the data does and does not support
- The candidate must have practical experience assessing and selecting guardrail methods, including rule-based controls, LLM-as-a-judge approaches and trained ML classifiers, with clear reasoning about their suitability and limitations in context
- The candidate must be able to translate a risk or expected behaviour into testable criteria and measurable controls, and to train or evaluate a classifier when that is the appropriate approach
- The candidate must have applied knowledge of precision and recall, detection thresholds, false positives, false negatives and class imbalance, and be able to balance safety, usability and operational constraints
- The candidate must have experience testing controls across realistic scenarios, critically checking assumptions and assessing how data or behaviour drift affects guardrail performance over time
- The candidate must be able to explain evaluation results, methodological choices and limitations to engineers, product managers and stakeholders
- The candidate must be able to work with use-case teams and domain experts to define what good performance means in context
- The candidate must be able to write clear technical proposals, evaluation reports and practical documentation
- The candidate must be able to turn ambiguous requirements into concrete experiments and implementation steps
- The candidate must be able to work in English
- The candidate must be available for the entire duration of the assignment
- The candidate must successfully complete the pre-employment screening before starting with the client
- The candidate must possess a valid passport or identity card that remains valid during both the intake and on the contract start date and can be presented for verification
- The candidate must not be submitted through a midlance employment construction
- The assignment is not suitable for ZZP professionals
- The candidate may only be submitted to one request at a time and, during the four-working-day exclusivity period, may only be submitted to Magnit - Rabobank
- The candidate's CV and personal motivation must be in English
- The personal motivation must describe how the candidate meets the required competencies
Wensen
- Experience with agentic evaluation, including tool selection, multi-step workflows and task completion is an advantage
- Experience testing privacy controls, prompt-injection defences or other GenAI safeguards is an advantage
- Experience with controlled adversarial testing or robustness assessment is an advantage
- Experience with evaluation tooling, observability or continuous evaluation in production is an advantage
- Experience working in a regulated environment and producing traceable validation evidence is an advantage