Presentation

Large language models are increasingly being used to code large corpora in the social sciences. Yet their probabilistic nature raises a fundamental problem: the same text and the same prompt can produce different responses. How, then, can we ensure the reliability and validity of the results?

Drawing on a project involving the semi-automated analysis of petitions in the Russian press, I propose to conceptualize the LLM as a probabilistic coder performing interpretive tasks. I will discuss the limitations of validation approaches based primarily on a human reference coding and present an alternative approach combining repeated coding, measures of stability, targeted human audits, and the explicit preservation of ambiguous cases. The aim is to treat model uncertainty as information to be analyzed, rather than simply as a flaw.

Guillaume Sauvé will be the speaker for this seminar.

Guillaume Sauvé is 2026-2027 fellow at the Paris Institute for Advanced Study. He teaches at the Université du Québec à Montréal and the Université de Montréal. His research lies at the intersection of comparative politics, political sociology, and intellectual history, with a particular focus on Russia in the late Soviet and post-Soviet periods. Drawing on archival research and interviews, he also incorporates computational social science methods.

The discussant will be Jérémy Robine, Associate Professor at Université Paris 8 and Director of the French Institute of Geopolitics.

More information

TETRIS Seminar

TETRIS (Terrains, Enquêtes et Techniques numériques pour l'étude des Récits et des Stratégies d'influence) is a research and training seminar devoted to methods for analyzing civilizational and strategic narratives, from their production and circulation to their reception, particularly through cyberspace infrastructures. It aims to develop “digital fieldwork” and to share these methods beyond academic circles.

Participation

The seminar is open to the public and can be attended either in person or online.
In-person venue: Campus Condorcet, Research Building North, Room 0.009
Registration: https://framaforms.org/seminaire-tetris-2-guillaume-sauve-le-codeur-probabiliste-1788358262