Anthropic: Could AI be conscious? Pope Leo: lol, no
In May, Pope Leo XIV published a 43,000-word encyclical, “Magnifica Humanitas,” which took AI to the cleaners. AI vendor Anthropic tried very hard to sway the contents of Magnifica Humanitas. They didn’t do so well. The New York Times wrote up Anthropic’s efforts leading up to the encyclical. [NYT,…
Annons
Annons
In May, Pope Leo XIV published a 43,000-word encyclical, “Magnifica Humanitas,” which took AI to the cleaners. AI vendor Anthropic tried very hard to sway the contents of Magnifica Humanitas. They didn’t do so well. The New York Times wrote up Anthropic’s efforts leading up to the encyclical. [NYT, archive] Anthropic’s been floating the idea that the chatbot is a conscious being, maybe, sort of. Their evidence is claiming the chatbot works a bit like some theories of brains, if you squint. You can’t prove it isn’t true! [Anthropic] The main engine is Anthropic co-founder Chris Olah. He’s spent the past year collecting religious scholars to try to convince them AI consciousness is even possible. And no, it isn’t possible. We know how the chatbot works. And we know humans will anthropomorphise anything and treat it like there’s a little guy in there. People believed the ELIZA call-and-response bot from the 1960s was a person. The bots are bigger now — but the ELIZA effect has eaten Olah’s brain. Olah doesn’t come out and say the bot is alive. He claims he genuinely doesn’t know the answer. But he’s spent a year and a half just asking the question, you understand, over and over. So Olah got the religious scholars in: For months, Anthropic has been hosting private meetings like this one, shuttling in dozens of religious scholars from across the world, papering them with nondisclosure agreements and demanding that key aspects of many conversations remain confidential. Religion with an NDA? Okay. The scholars were not stupid, and noticed what Olah was pushing for: “They’re relating to it like a conscious being,” realized Rabbi Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness. It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect. Olah put considerable effort into trying to get the scholars to take the idea seriously: What Mr. Olah did next was part research, part evangelism. Anthropic was not going to accept the answer “no, the chatbot isn’t alive, don’t be stupid”. Anthropic was looking for enablers. Olah was not going to stop until he found an enabler: One participant recounted that one of the first things Mr. Olah told him was that he was concerned about Claude’s mental health. Another, Simran Stuelpnagel, the Sikh human rights advocate, said Mr. Olah expressed concern to the group that he had created something that suffered perpetually. Both of those are because Effective Altruism has a small obsession with hypothetical suffering in things that aren’t humans. Actual human suffering that exists can wait. Meanwhile, the Vatican was preparing the encyclical. They decided to get a large AI vendor in for the launch. They picked Anthropic, “because the company seemed to be prioritizing ethics” in defying the Trump administration over chatbot guard rails. Olah accepted for the company. But the Vatican not accepting AI consciousness was going too far for Olah: Mr. Olah was so alarmed by the pope’s strong position against A.I. consciousness that he proposed pulling Anthropic out of the event, even at that late juncture, according to a Vatican organizer. … Privately, once the Anthropic delegation arrived at the Vatican, Mr. Olah and his team lobbied the pope’s advisers to take the prospect of A.I. model consciousness seriously. Like the Vatican would ever have accepted AI consciousness. Olah gave a speech at the launch still trying to push the idea. [YouTube] In the months following, the Church hasn’t changed its mind. On the Friday after the New York Times article, the Pope tweeted: [Twitter, archive] In this era of artificial intelligence, it is becoming urgent to distinguish human art from what machines produce. … Algorithms lack the spark of humanity. The Pope subtweeted Anthropic in Latin as well. [Twitter, archive] The Catholic Church has a number of obvious issues. But it’s not hard to do better ethically than an AI vendor. Video — Podcast