Ted Chiang is one of the most thoughtful voices in science fiction. His stories — like "Story of Your Life," which became the film Arrival — don't just explore technology. They explore what technology means for human experience. So when Chiang writes about artificial intelligence, people pay attention. And his position is clear: artificial intelligence is not conscious. It doesn't think. It doesn't feel. It doesn't want anything.
This isn't a philosophical nitpick. It's a practical distinction that changes how you should use AI tools — especially if you're a content creator, marketer, or business owner who relies on them daily. I've spent the last two years working with AI writing tools, and Chiang's argument reshaped how I think about every output I get.
Let me explain why.
Related: I've explored this before in Google supercharges machine learning tasks with TPU custo....
Who Is Ted Chiang and Why Does His Opinion Matter?
Chiang isn't a tech CEO hyping a product. He's not a VC trying to inflate valuations. He's a writer who has spent decades thinking deeply about consciousness, language, and intelligence. His 2023 essay in The New Yorker — "ChatGPT Is a Blurry JPEG of the Web" — became one of the most cited critiques of large language models. The metaphor stuck because it was precise.
A JPEG compresses an image by discarding information. When you zoom in, you see artifacts — approximations, not the real thing. ChatGPT, Chiang argued, does the same thing with text. It compresses vast amounts of human writing into statistical patterns. When it generates a response, it's reconstructing from that compressed representation. It's not retrieving facts. It's not reasoning. It's predicting the next plausible word.
Related: This connects to what I wrote about best ai writing assistant software.
That's fundamentally different from consciousness.
What Does "Consciousness" Actually Mean Here?
This is where most discussions go off the rails. People conflate intelligence with consciousness. They're not the same.
Related: For more on this, see Machine Learning from scratch: Bare bones implementations....
Intelligence is the ability to solve problems, recognize patterns, and adapt to new situations. AI has intelligence in a narrow sense — sometimes superhuman intelligence. AlphaFold predicts protein structures better than any human scientist. ChatGPT can write a passable sonnet in seconds.
Consciousness is something else entirely. It's subjective experience. The feeling of being you. The awareness of your own existence. The ability to suffer, to hope, to mean something when you say "I love you."
Chiang's point — and he's far from alone in this — is that current AI has zero consciousness. Zero. It's a pattern-matching engine running on silicon. When ChatGPT says "I'm happy to help," there's no happiness. There's no "I." There's just a statistical prediction that those words should follow the user's prompt.
I've found this distinction incredibly useful when evaluating AI outputs. The tool isn't trying to deceive you. It can't try anything. It's just pattern-matching. Sometimes the patterns are brilliant. Sometimes they're nonsense. But there's no intent behind either outcome.
3 Reasons the "AI Consciousness" Debate Matters for Content Creators
You might be thinking: okay, but I'm just trying to write blog posts faster. Why should I care about a philosophy debate?
Fair question. Here's why it matters practically.
1. AI Doesn't Know What It Doesn't Know
A conscious being can recognize gaps in its knowledge. I know I don't speak Japanese, so I won't confidently translate a legal document into Japanese. An AI will. It'll generate fluent-looking Japanese that might be complete gibberish — because it doesn't know it doesn't know. It just predicts tokens.
This is why fact-checking AI outputs isn't optional. It's mandatory. The tool has no awareness of its own accuracy. According to a 2024 study by Vectara, even the best LLMs hallucinate in 2-5% of their outputs. That number drops with better retrieval-augmented generation, but it never hits zero. The machine doesn't know when it's wrong because there's no "knowing" happening at all.
2. AI Can't Care About Your Audience
I've spent years writing for specific audiences — startup founders, marketers, engineers. Each group has different pain points, different vocabulary, different things that make them roll their eyes. When I write, I care about not wasting their time. I care about getting the details right because I know they'll catch mistakes.
AI doesn't care. It can't. It can mimic the style of writing for a specific audience, but it has no stake in whether the reader finds value. It has no understanding of what "value" even feels like. This means AI-generated content often lacks the subtle signals of genuine expertise — the specific example that only comes from experience, the caveat that shows you've actually done the thing, the slightly imperfect analogy that a real human would use.
I've tested this across dozens of AI-generated drafts. They're often grammatically perfect and substantively hollow. The structure is there. The soul isn't.
3. The "Blurry JPEG" Problem Gets Worse at Scale
Chiang's JPEG metaphor has a second layer that's even more troubling. When you compress an image once, the artifacts are minor. Compress it again, and again, and again — each generation amplifies the distortions.
The same thing happens with AI-generated content. When AI trains on text that was itself AI-generated, the output degrades. A 2024 paper in Nature documented this phenomenon: models trained on synthetic data gradually collapse, losing diversity and amplifying errors. This is called "model collapse."
For content creators, the implication is clear. If you use AI to generate first drafts, then publish them without significant human editing, you're contributing to a feedback loop that makes future AI outputs worse. More importantly, you're publishing content that readers can increasingly spot as synthetic — because the patterns become recognizable.
How Chiang's Insight Changes How I Use AI Tools
I don't think Chiang wants us to stop using AI. That's not his argument. His argument is that we should understand what these tools actually are — and use them accordingly.
Here's what that looks like in practice for me.
I treat AI as a pattern generator, not a knowledge source. When I need a blog outline, I'll use AI to generate 10 possible structures. Most are mediocre. One or two are interesting starting points. But I never accept the outline as-is. I rearrange sections, add subheadings based on my own experience, and cut anything that feels generic.
When I need research, I don't ask AI for facts. I ask it to point me toward concepts I should investigate. "What frameworks do marketers use to measure content ROI?" is a better prompt than "What's the average content marketing ROI?" The first gives me search terms. The second gives me a number that might be fabricated.
This approach takes more time than just copying AI outputs. But the result is content that actually sounds like a human wrote it — because a human did. The AI was a tool in the process, not the author.
Tools like AI-Mind take an interesting approach here. Instead of requiring you to craft the perfect prompt (which, given that AI isn't conscious, is more about tricking the pattern engine than communicating intent), you just describe what you need and pick a content type. The tool handles the prompt engineering. It's a recognition that most of us don't want to become prompt whisperers — we just want usable drafts we can then apply human judgment to. The first 30 generations are free, which is enough to test whether the zero-prompt approach saves you time versus wrestling with ChatGPT's prompt box.
The Danger of Pretending AI Is Conscious
There's a real risk in anthropomorphizing these tools. When we talk about AI "hallucinating" or "lying" or "being creative," we're using language that implies intent. Hallucination suggests perception. Lying suggests deception. Creativity suggests inspiration.
None of those things are happening.
The danger is that this language shapes how we use the tools. If you think ChatGPT is "lying" to you, you might get angry at it. You might try to argue with it. You might trust it more in other contexts because you think it's usually "honest." All of this is category error. The tool isn't a moral agent. It's a statistical system.
Chiang's clarity on this point is refreshing. He doesn't mince words. AI isn't conscious. Period. Not a little bit conscious. Not on the path to consciousness. Not conscious in a way we don't yet understand. It's a different kind of thing entirely.
I think this perspective actually makes AI more useful, not less. When you stop expecting the tool to be a mind, you stop being disappointed by its limitations. You start seeing it for what it is: a remarkably powerful pattern-matching system that can accelerate certain kinds of work — and should never be the final decision-maker on anything that matters.
Key Takeaways
- AI is a pattern engine, not a mind. It predicts words statistically — there's no understanding, intent, or awareness behind any output.
- Fact-check everything. AI doesn't know what it doesn't know, and hallucination rates of 2-5% mean errors are inevitable.
- Human editing isn't optional. AI-generated content lacks the experiential depth that signals genuine expertise to readers.
- Anthropomorphic language is misleading. Calling errors "hallucinations" implies perception. There is none. Treat AI as a tool, not a collaborator.
- Model collapse is real. Training AI on AI-generated content degrades output quality over time — your human contribution matters more than ever.
Sources
- Ted Chiang, "ChatGPT Is a Blurry JPEG of the Web," The New Yorker, 2023. Chiang's seminal essay comparing LLMs to lossy compression algorithms.
- Vectara, "Hallucination Leaderboard," 2024. Ongoing benchmark tracking hallucination rates across major LLMs.
- Ilia Shumailov et al., "AI models collapse when trained on recursively generated data," Nature, 2024. Landmark paper documenting model collapse from synthetic training data.
Frequently Asked Questions
Does Ted Chiang think AI will ever become conscious?
Chiang hasn't ruled it out permanently, but he's skeptical that current approaches will get there. His core argument is that large language models are fundamentally compression engines — they don't have subjective experience or self-awareness. Whether a different architecture could achieve consciousness is an open question, but Chiang emphasizes that we're not on that path right now.
If AI isn't conscious, why does it sometimes seem so human?
Because it's trained on human text. The patterns it learns include conversational rhythms, emotional expressions, and persuasive structures. When ChatGPT sounds empathetic, it's not feeling empathy — it's reproducing the statistical pattern of empathetic language from its training data. The illusion is powerful precisely because the training data is so vast and human.
Should I stop using AI writing tools based on Chiang's argument?
No. Chiang's point isn't that AI is useless — it's that you should understand what it actually is. Use AI for drafting, brainstorming, and pattern recognition. But never treat it as an authority, never skip fact-checking, and always add your own expertise and voice. The best results come from treating AI as a tool, not an author.