Eli Pariser, author of The Filter Bubble, reads Leo XIV’s Magnifica humanitas: aligning the incentives of those who build AI with the common good.
The post Magnifica humanitas, Leo XIV’s encyclical read by Eli Pariser: “Disarm the economic incentives behind AI” first appeared on AgenSIR.“If we can align the incentive structures of the builders with the public good and the common good, that is the most critical task”. Eli Pariser, author of The Filter Bubble and co-director of New_ Public, a US organisation that studies and designs digital spaces able to sustain civic life, gives SIR his reading of the first encyclical of Pope Leo XIV, Magnifica humanitas, signed on 15 May and dedicated to safeguarding the human person in the time of artificial intelligence. From the hyper-personalisation of generative AI to the images of Babel and Jerusalem, from the opacity of the logic that governs the models to the possibility of a different economy: the Magisterium in dialogue with the view of one of the best-known critics of the platforms.
Years ago you coined the expression “filter bubble”. With generative AI systems, are we facing a new kind of bubble, or something even more radical?
Generative AI enables a level of personalization of information that I couldn’t have imagined when I wrote The Filter Bubble. AI can draw on deep personal context – our emails, our music histories, our documents and personas, the ways we interact – to tailor information to our beliefs and personality traits, and to do so interactively and over time.
So there is a risk that if these technologies are built to flatter us, to be sycophantic, they could distort what we know.
And if they are co-opted by others who seek to influence us, or by economic actors who want to sell us things, they could be very persuasive in that regard.
Magnifica humanitas does not ask, first of all, whether artificial intelligence is good or bad, but what idea of the person is embedded in the technology. Is this the right starting point?
I do think this is the right starting point. Technological discourse often centers the technology. But as the encyclical says, it’s critical to center humanity and to ask: how is this serving us? People have a lot of worries, and well-founded ones, that this technology is not really built around serving people – around making our lives better, or our jobs more fulfilling, or our families more connected. So asking who this is for, what it is for, and what version of humanity it is meant to serve seems to me an excellent starting point.
Pope Leo XIV uses two images: Babel and Jerusalem. On one side, technology as domination and uniformity; on the other, rebuilding as shared responsibility. Can this framework also speak to people outside a religious perspective?
As someone new to this set of metaphors, I found them vivid and persuasive – partly because both are stories about human construction, about building things, that play out in very different ways: one centralized and dominating, the other decentralized and shared. In my work, we come up against these two tendencies often. I have spent a lot of time thinking about social media, and one of my critiques has been that it’s really impossible to come up with an algorithm that will work for billions of people and millions of communities around the world. Human society is too varied, too diverse, too multifaceted for that.
We need systems that are more in the control of communities, built from the ground up, like Jerusalem, rather than architected centrally, like Babel.
We’re starting to see that in places like Bluesky, and with systems like Signal, that are built on open protocols and driven by public values rather than private ones. I think there’s momentum for the Jerusalem way of building technology.
You have argued that many digital platforms are not conceived to serve the community but for advertising, profit and engagement. Does AI risk replicating the same model?
Certainly that’s the path we’re on right now. These technology companies – Anthropic, OpenAI, Google – have all spent billions of dollars, in many cases other people’s money, and those people expect to make their money back. They’re going to do it by capturing value back from users, which is where things go bad in technological products.
But I want to emphasize that it doesn’t have to be that way.
There is nothing intrinsic to this technology that means it has to be driven by advertising or by profit. In fact, we’re seeing the growth of an open source ecosystem that offers some really different models for how value might be generated. So I think we don’t know yet, and it’s really important to maintain a sense that there is a choice ahead of us. The choice has not been made yet about how we want to economically integrate and build these systems.
At the presentation of the encyclical, the Pope asked that this technology be “disarmed”, “freed from logics that turn it into an instrument of domination, exclusion and death”. What should be disarmed first: the machines, the economic incentives that sustain them or the way they shape public language?
I would argue the economic incentives. When we look at tech, the economic incentives are such a core driver. That’s certainly where social media went off the rails – as it started monetizing through advertising, and weaponizing people’s personal data to sell them things. So if we can align the incentive structures of the builders with the public good and the common good, that is the most critical task.
If you had to name one concrete choice to steer algorithms towards truth, human dignity and the common good, what would it be?
One thing I’m very interested in is helping people understand and audit the incentive structures behind the AI they use.
Right now it’s very unclear – totally opaque – why a given AI service is showing me something.
I don’t know whether I’m being manipulated, how the data being fed to the model was gathered or developed, or who might be trying to persuade or influence me. So making it easier for people to understand how these systems are constituted, and what the incentive structures and intent behind them are, seems to me like a very good place to start.
The post Magnifica humanitas, Leo XIV’s encyclical read by Eli Pariser: “Disarm the economic incentives behind AI” first appeared on AgenSIR.