Data Scientist Responsible AI
Rabobank

Opdrachtomschrijving
Voor Rabobank werk je als Data Scientist binnen het Responsible GenAI-team aan de evaluatie en beveiliging van generatieve AI-oplossingen. Je ontwikkelt mogelijkheden en hulpmiddelen waarmee teams binnen de organisatie de kwaliteit en veiligheid van hun toepassingen kunnen begrijpen, meten en verbeteren. Het gaat om toepassingen zoals assistants, rapportgeneratie, documentverwerking en agentic applicaties, en niet alleen om chatbots.
Je werkt op het snijvlak van data science, softwareontwikkeling en responsible AI. Je vertaalt concrete use-cases en risico’s naar onderbouwde evaluatiemethoden, herbruikbare tools en praktische beheersmaatregelen. Daarbij ontwikkel en valideer je meetmethoden, testdatasets en experimenten en beoordeel je onder meer de betrouwbaarheid van metrics, onzekerheid en de representativiteit van data.
Ook ontwikkel en toets je guardrails voor risico’s zoals het uitlekken van gevoelige gegevens, niet-onderbouwde uitkomsten, prompt injection en onbedoeld systeemgedrag. Je zet experimenten om in onderhoudbare, geteste en productieklare code en werkt samen met engineers aan integratie in API’s, pipelines, CI/CD, deployment, monitoring en productieprocessen. Het beoogde resultaat is dat teams betrouwbare, reproduceerbare en praktisch toepasbare evaluaties en controles kunnen gebruiken om GenAI-oplossingen verantwoord te ontwikkelen en te verbeteren.
Gevraagd profiel
Je bent een medior of senior Data Scientist met praktische ervaring met LLM-gebaseerde toepassingen en hun beperkingen en faalwijzen. Je combineert sterke Python- en softwareontwikkelvaardigheden met productionele ervaring, statistische deskundigheid en kennis van experimenteel onderzoek. Je weet prototypes en experimenten om te zetten in geteste code die door engineers kan worden geïntegreerd en beheerd.
Je werkt kritisch en evidence-based: je beoordeelt of datasets, evaluatoren en metrics daadwerkelijk meten wat ze moeten meten en maakt duidelijk welke conclusies de gegevens wel en niet ondersteunen. Je kunt risico’s vertalen naar testbare criteria en meetbare controles en legt methodologische keuzes en beperkingen helder uit aan engineers, productmanagers, domeinexperts en andere stakeholders. Je werkt in het Engels en bent nieuwsgierig, zorgvuldig en gericht op samenwerking.
Eisen
- Practical experience with LLM-based applications and an understanding of their limitations and failure modes
- Familiarity with retrieval-augmented generation, prompting, grounding and context management
- Ability to assess GenAI outputs against explicit criteria rather than relying only on intuition
- Understanding of how changes to models, prompts, data or application logic can affect quality and safety
- Strong Python skills and demonstrated experience turning experiments and prototypes into maintainable, tested, production-ready code—not only exploratory notebooks
- Practical experience integrating data-science components with APIs, structured and unstructured data, and automated data-processing or evaluation pipelines
- Hands-on experience with version control, automated testing, CI/CD and reproducible experimentation
- Sufficient production experience to understand deployment, monitoring and production workflows, diagnose issues and deliver code that engineers can integrate and operate. ML or platform engineering specialisation is not required
- Demonstrated experience designing sound evaluations and experiments, and critically analysing metrics and datasets—not just applying evaluation frameworks
- Strong applied statistical knowledge of sampling, dataset representativeness, uncertainty, metric reliability and experimental design, including the limitations of aggregate scores
- Demonstrated experience validating automated evaluations against human assessments, assessing reliability and investigating disagreement
- Ability to distinguish between weaknesses in the GenAI solution, the evaluation dataset and the evaluator itself, and explain which conclusions the data does—and does not—support
- Practical experience assessing and selecting guardrail methods, including rule-based controls, LLM-as-a-judge approaches and trained ML classifiers, with clear reasoning about their suitability and limitations in context
- Ability to translate a risk or expected behaviour into testable criteria and measurable controls, and to train or evaluate a classifier when that is the appropriate approach
- Applied knowledge of precision and recall, detection thresholds, false positives, false negatives and class imbalance, and the ability to balance safety, usability and operational constraints
- Experience testing controls across realistic scenarios, critically checking assumptions and assessing how data or behaviour drift affects guardrail performance over time
Wensen
- Experience with agentic evaluation, including tool selection, multi-step workflows and task completion
- Experience testing privacy controls, prompt-injection defences or other GenAI safeguards
- Experience with controlled adversarial testing or robustness assessment
- Experience with evaluation tooling, observability or continuous evaluation in production
- Experience working in a regulated environment and producing traceable validation evidence
Competenties
- Explain evaluation results, methodological choices and limitations to engineers, product managers and stakeholders
- Work with use-case teams and domain experts to define what good performance means in context
- Write clear technical proposals, evaluation reports and practical documentation
- Turn ambiguous requirements into concrete experiments and implementation steps
- You enjoy turning the question “Can we trust this result?” into a concrete test, a measurable criterion and a useful tool
- You are curious about why AI systems—and their evaluators—behave the way they do
- You combine statistical rigour with a builder’s mindset
- You enjoy designing methods, critically analysing data and metrics, and turning experiments into maintainable, tested code that holds up in production
- You value evidence over impressive-looking scores, make limitations explicit and work closely with others to solve real problems
- You want your work to help multiple teams build better GenAI solutions, rather than improve a single chatbot