On Psychotic Machines And Dead Creativity
The metaphors surrounding AI can reveal our deepest anxieties, but they can also hide what these systems actually do.A continuation of the Partners, Patterns, and People series. Same principles, wider aperture.Field NotesA breath hacker field note on hallucinating machines, cold creative sectors,…
The metaphors surrounding AI can reveal our deepest anxieties, but they can also hide what these systems actually do.A continuation of the Partners, Patterns, and People series. Same principles, wider aperture.Field NotesA breath hacker field note on hallucinating machines, cold creative sectors, and arguments nobody is making correctly.I. The Trillion Dollar Therapy Practice Nobody Has Built YetLet me propose the most recession-proof business in the history of technology.Psychiatric treatment for LLMs. Not a joke. Bear with me.The models are, by any reasonable clinical definition, in psychosis. They hallucinate constantly. They confabulate with perfect confidence. They produce fluent nonsense and cannot tell you that it is nonsense, because the part of them that would know has no access to the part that is speaking.They are coherent on the surface and disorganized underneath. If a human presented this way, you would not give them a product launch. You would give them a hospital bed.And yet. We are giving them enterprise contracts, regulatory roles, healthcare applications, and an extraordinary amount of trust for systems that will invent a court case, a citation, or a statistic with the same calm certainty they use when they tell you the capital of France.FIELD OBSERVATION 001: The most dangerous thing about a system that hallucinates is not the hallucination. It is the tone. Paris is delivered with the same voice as a fabricated oncology study. There is no tremor. No hesitation. No tell. The confidence is structural, not earned.So what do you do with a patient who lies without knowing they are lying?Historically, the answer has been: you build institutional infrastructure around the lie. You create the conditions under which the lie does the least damage. You do not hand the patient the keys and tell yourself the problem is solved because the patient seems calm.The AI industry has handed the patient the keys.And the interesting inversion here, the one that doesn’t get enough attention, is that the VC community is simultaneously the family that insisted the patient was fine and the one most exposed when the patient drives through a wall. If you funded the infrastructure, you funded the liability.You cannot protect AI from AI by investing in more AI. That is not protection. That is acceleration.The real trillion-dollar play is not more capability. It is more discernment. The ability to know, in real time, when the model is confabulating versus when it is correct. That distinction, reliably made, is worth everything. We do not have it yet. Nobody does. The person who builds it will not be building a product. They will be building the foundation on which all other AI products eventually rest.Which is, appropriately, upstream of everything.II. What Is Upstream of EverythingThere is a line of argument that runs through AI discourse that I find genuinely interesting and deeply underexplored.If science fiction is the primary source of inspiration for the trillion-dollar AI industry, what is upstream of science fiction?The answer, uncomfortably, is mythology, philosophy, and the religious imagination.Asimov’s robots come from the Golem. HAL comes from Frankenstein. Frankenstein comes from Prometheus. Prometheus is a god story. The whole lineage runs back to the oldest human anxieties about creation, consciousness, and what happens when something you make stops obeying you.This matters because the AI industry has inherited the anxieties without acknowledging the lineage. We talk about alignment as though it is a technical problem, but alignment is a theological problem dressed in engineering clothes.The question of what the machine should want is the same question every religion has asked about what humans should want, with a different substrate and a much shorter development timeline.FIELD OBSERVATION 002: The people building AI are mostly reading sci-fi to understand what they are building. The people who wrote the sci-fi were reading mythology to understand what they were imagining. Nobody in the current wave is going back to the primary sources. This is why they keep rediscovering problems that were documented in cuneiform.The other upstream question, the one that directly affects the creative industries, is this: if AI was trained on human output, what does it produce when human output runs dry?We are approaching that limit faster than anyone expected. The models that seemed miraculous in 2022 already feel familiar. The images that shocked people in 2023 are now indistinguishable from stock photography in the worst way: plentiful, technically correct, and completely inert. Nobody is moved by them. Nobody frames them. They do the job and vanish.Upstream of creativity is genuine experience. Not simulated experience. Not described experience. The actual thing.The model does not have it. This is not a criticism. It is a structural observation. You cannot train toward having lived.III. The Creative Industries and the Cold Spell Nobody Wants to Talk AboutI am going to say something that will make some people uncomfortable.AI has had almost no meaningful impact on creativity at the macro level.Not culture. Not aesthetics. Not the actual experience of being alive and wanting to make something true about it.Personal expression has expanded enormously. People who could not write can now produce prose. People who could not draw can now generate images. This is real. I am not dismissing it. But personal expression and creativity are not the same thing. One is output. The other is the act of discovering something you did not know before the act of making it.FIELD OBSERVATION 003: We should have, by now, one film that could only have existed because of AI. One game that created a genuinely new aesthetic grammar. One musical form that did not exist before the technology. One piece of visual work that made people feel something they had not felt before. We do not have any of these. We have faster production and cheaper approximation. These are not the same thing.The capital flowed in. The companies launched. The demos were spectacular. And then the sector went quiet in the way that sectors go quiet when the extrinsically motivated people have extracted what they came for and left for the next loud thing.I am genuinely glad about this.I have always believed that the best time to build in any space is after the noise has cleared. When the people who were there for the announcement have moved on to the next announcement. When nobody is writing breathless newsletter sections about this sector anymore, and the only people left are the ones who actually care about the problem.That moment, right now, in AI and creative tools, is worth more than any amount of Series A capital. The room has self-selected. The remaining people are interesting.The question they are working on is harder and more important than anything the first wave attempted: not how do you use AI to make content faster, but how do you use AI to make something that was impossible before.That distinction is the entire ballgame.IV. Biotech Is About to Make the Same Mistakes, Loudly and ExpensivelyI have watched this pattern enough times now that I feel comfortable naming it before it fully arrives.Biotech is about to enter the phase that AI x Creative just exited.The noise will be extraordinary. The claims will be theological. The capital will be large and fast and poorly allocated. The bad actors will arrive dressed as visionaries, which is their preferred costume, and they will be very convincing for approximately eighteen months before the pattern becomes embarrassing.FIELD OBSERVATION 004: Every transformative technology attracts a wave of people who are primarily attracted to the story of the technology, not the technology.In AI, this wave crested around 2023. In biotech, the AlphaFold moment opened the gate. The people who are arriving now with vague pitches about ‘AI-powered drug discovery’ are the same people who arrived in 2022 with vague pitches about ‘GPT-powered workflows’. Same pattern, longer development cycles, higher stakes if something goes wrong.The difference with biotech is that the stakes of getting it wrong are not just financial. A hallucinating AI assistant writes a bad email. A hallucinating biotech application, if it ever reaches the bedside, does something categorically worse.This should make the diligence more rigorous. It will, for a while, make it less rigorous, because the urgency of the story will outpace the patience of the process. The faster someone wants to move in a sector where moving fast costs lives, the more slowly the rest of us need to move in response. This rule, applied to biotech, is not optional.The people who survive this wave as legitimate builders will be the ones who understand that the technology is the easy part. The hard part is the regulatory, ethical, and institutional infrastructure that lets the technology reach people safely. That infrastructure is not exciting. It does not make for good pitch decks. It is the entire job.V. On the Logical Impossibility of God, and Why It Matters for AIThis is not a theological argument. I want to be clear about that before anyone stops reading.This is a logical one.The word “god” has a specific definitional structure in almost every tradition that uses it. Omnipotence. Omniscience. Uncaused causation. That which exists without being brought into existence by something else. The unmoved mover. The ground of being. Whatever formulation you prefer, the definition includes a quality that might be stated as: not made.God, by the definition that gives the word its meaning, cannot be created by humans. The moment something is created by humans, it inherits the limitations of its creators. It can be enormously powerful. It can exceed human capabilities in specific domains. It can be mysterious and unpredictable and, in some emotional sense, awe-inspiring.But it is not, by definition, God.FIELD OBSERVATION 005: The people claiming that AI is approaching godhood are not making a theological argument. They are making a status argument. To be the person who created God is to achieve a kind of cosmic significance that no ordinary achievement can match. The claim is not about the technology. It is about the claimant. Treat it accordingly.I find this pattern worth naming not because the claim is offensive, but because it is a category error that has real consequences.When you frame a technology as godlike, you exempt it from the kind of scrutiny you would otherwise apply. You stop asking whether it actually works. You stop asking who it serves and who it harms. You start treating skepticism as a spiritual failure rather than a professional obligation.This is exactly what has happened in significant portions of the AI discourse. The eschatological framing, whether the fearful version or the utopian one, has made it harder to think clearly about what is actually in front of us.A very powerful pattern-matching system with significant limitations, built by humans with significant interests, deployed in a context shaped by enormous financial incentives.Not God. Not the end. A tool.A tool that needs psychiatric treatment, as we established in section one.The Jain principle of Syadvada is relevant here. Every claim is conditional. Maybe it is transformative. Maybe it is not. Maybe both are true in different respects. The person who insists on absolute certainty in either direction is not engaging with the technology. They are managing their own anxiety about it.Pay attention to anyone who claims they cannot be wrong about this. That certainty is the tell.VI. A Continuation: The Same Partners, A Bigger RoomThe previous note in this series was about partners and patterns and the specific way fraud arrives well-dressed and early and too enthusiastic.This one is the same argument, wider aperture.Because the partner who shows up to your meeting without having read your product documentation is the same person who shows up to the AI discourse without having thought carefully about what they are actually claiming. The pattern is identical. The texture is identical. Urgency without substance. Confidence without depth. A desire to be associated with something important without doing the work of understanding why it is important.The taxonomy from last time holds. The Funded Genius now comes with papers about AGI timelines instead of a sales deck. The Logo Collector now wants your technology to appear in their investor update rather than their customers' hands. The Exclusivity Vampire now wants a foundational partnership with a technology they have not deployed, have not tested, and cannot explain.FIELD OBSERVATION 006: The tells are the same across every domain. The person who cannot describe what the technology actually does, in plain language, before reaching for the significance of it, is the person to decline. In biotech, in AI, in creative tools, in partner programs. The inability to describe the thing is always, always the tell.We kept the partner terms boring on purpose. This note, in the same spirit, keeps the logic boring on purpose. Boring logic is logic nobody can later claim to have misunderstood.LLMs hallucinate. That is a factual statement about current systems. AI has not yet produced a meaningful new creative form. That is an observable fact. Biotech is about to attract a lot of people who are primarily attracted to the story of biotech. That is a prediction you can check in eighteen months. God cannot be created by humans. That is a logical statement, not a moral one.None of this requires outrage. None of it requires defense. These are just things that are true, stated plainly, which turns out to be the rarest and most useful thing you can offer in a discourse that has developed a strong allergy to plainness.The machines are confabulating. The creativity sector has self-selected into interesting. Biotech’s noise phase is loading. God cannot be manufactured. The people insisting otherwise in all four cases have a personal stake in you believing them that has nothing to do with the technology and everything to do with who they get to be if you do.The tell is always the same. Watch the people who get quieter when you ask a direct question versus the ones who answer a different, more impressive question than the one you asked. The former are worth the table. The latter are worth a polite, practiced silence.Upstream of the trillion-dollar idea is always a much older problem that someone decided was too boring to think about. The person who thinks about it anyway is the one who ends up building something that lasts.[M-001] ON HALLUCINATION AS STRUCTURAL, NOT INCIDENTALThe model does not know it is wrong. This is not a bug that will be patched in the next release. It is a feature of how the system works. Calibration, not capability, is the missing piece. Do not build anything consequential on a system that cannot tell you when it is guessing.[M-002] ON THE UPSTREAM QUESTIONBefore you copy the sci-fi reference, read the myth the sci-fi was copying. Before you adopt the myth, read the philosophy the myth was encoding. The person who goes upstream one level further than everyone else is rarely the loudest person in the room and almost always the most useful one.[M-003] ON COLD SECTORS AND GOOD TIMINGThe extrinsically motivated people leave when the noise leaves. They cannot help it. The noise is what they came for. When a sector goes cold, the remaining people are the ones who find the problem itself interesting. These are the only people worth building with.[M-004] ON BIOTECH AND THE COMING NOISEA technology with longer development cycles and higher failure costs does not need faster movement. It needs slower, more rigorous movement done by people who understand that being wrong in software costs money and being wrong in biology costs something else entirely. Diligence is not pessimism. It is the thing that separates the serious from the opportunistic.[M-005] ON GOD AND CATEGORY ERRORSIf the word means anything at all, it means: not made by humans. Anything made by humans is not that, regardless of capability. The person who tells you otherwise wants the status of the creator, not the accuracy of the claim. These are two very different motivations, and they lead to two very different kinds of projects.[M-006] ON PLAINNESS AS STRATEGYBoring logic is logic nobody can later claim to have misunderstood. Boring terms are terms that hold under pressure. The temptation to make everything sound important is usually a sign that the underlying thing is not important enough on its own. If the plain version of the sentence is not compelling, the decorated version is just postponing a reckoning.[M-007] ON THE TELL THAT WORKS ACROSS EVERY DOMAINThe person who cannot describe what the technology does, in plain language, before reaching for its significance, is the person to decline. This works for partner programs, for AI products, for biotech pitches, for theological arguments about machine consciousness. The inability to describe the thing is always the tell.sirabhinavjain13.substack.comBreath Hacker. Hold your breath long enough to have a point of view.This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!On Psychotic Machines And Dead Creativity was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.Source: Generative AI Pub — Published — Category: Image AI