ChatGPT Faked a Deep Read of My Novel

How AI Confidently Hallucinated an Entire Plot

For grins and giggles, I asked ChatGPT to tell me about my novel, Threading the Needle, via its Deep Research function. I really was not expecting to be writing about it or using it to show you the limitations of LLMs.

The Deep Research functionality is described by Google as:

Deep research is a specialized AI capability that uses extensive browsing and analysis of online sources to generate detailed reports on complex topics. It's designed to go beyond basic question answering by performing multi-step research, synthesizing information, and presenting findings in a comprehensive and well-cited manner. Essentially, it's an AI assistant that can help you explore a topic in depth, gathering information from various sources and compiling it into a usable report.

I keep telling people that LLMs don’t and can’t think, yet some insist that they can and turn to ChatGPT for everything.

I told the LLM that I wanted to be able to discuss the book as if I’d read it. I wanted a chapter-by-chapter breakdown and a focus on characters and their arcs, the worldbuilding details, and the themes in the book.

ChatGPT: Understood. I’ll begin compiling a detailed chapter-by-chapter summary of Threading the Needle by Monalisa Foster, including character arcs, political and technological worldbuilding, and core themes such as freedom vs. control and tradition vs. progress. This will prepare you to discuss the book confidently as if you've read it. I’ll let you know as soon as the full analysis is ready.

So off it went to look stuff up, starting with Baen’s website, and then the various reviews available online. Notice how it had no objection to me asking for a chapter-by-chapter, something that couldn’t be parsed without full access to the manuscript. It did not clarify that it had no access to that, or if it did, how. It blithely ignores the issue even though it does have policing/moderating functions, as I’ve shown before.

The policing functions aren’t there to police “AI” but to police you, what you can say, how you can say it, etc. But I digress…

Some time later, Deep Research returned with an extended treatise (it is very verbose) but we won’t be going over all of it because Substack has a post-length limit.

It begins with a summary:

ChatGPT: Monalisa Foster’s Threading the Needle is a military sci-fi meets space Western set on the colony world of Gōruden. The novel follows Talia Merritt, an ex-sniper with a cybernetic arm, as she seeks a peaceful new life but becomes entangled in a struggle between free‐spirited colonists and a manipulative corporate overlord. Below is a chapter-by-chapter breakdown, including key plot events, character developments, worldbuilding details, and thematic elements, to prepare you to discuss the book as if you’ve read it.

So far so good.

Do know that I don’t title my chapters. I number them. So the fact that ChatGPT titles them is the first betrayal of its generative (it makes stuff up) nature.

ChatGPT: Chapter 1: A New Beginning on Gōruden

Major Plot Events: Talia Merritt arrives on Gōruden to start a new life, hoping to leave behind her bloody past. We learn she has finally earned her freedom after seven years of servitude – she had been indentured as a bodyguard to pay for her passage and stake on this colony (baen.com). Now free, she travels to the frontier town of Tsuri for a job opportunity as a firearms instructor. The chapter establishes Talia’s determination to make a fresh start and hints at the “new start or old calling” dilemma she will face (baen.com).

It’s Tsurui, not “Tsuri” a typo that made it into the ad-copy and is carried throughout, so we know it didn’t actually ever look at the manuscript here. And to be clear, it should not be able to look at the manuscript at all. Why?

Because of copyright. The copyright statement is very clear:

All rights reserved, including the right to reproduce this book or portions thereof in any form.

It forbids copying (i.e. reproduction) of any kind and in order for the LLM to answer any questions about this story, it would have to make a copy for itself. Even if that copy exists only for one microsecond, it has violated copyright. I would expect the LLM to have access to what’s on the Baen website, the free previews, and other online sources, but not the full manuscript itself. If it does, it’s violating copyright, no matter how it uses it.

Nevertheless, ChatGPT still attempted to answer and in a very confident matter, very much like a student who never did the work but still has to bluff their way through an oral exam or a pop-quiz.

ChatGPT: Character Introductions & Development: Talia Merritt is introduced as a war veteran haunted by her past deeds. Once known by the chilling moniker “Death’s Handmaiden” for her lethal record as a sniper, she lost an arm in combat and lives with a cybernetic prosthetic arm controlled by a neural implant (her “phantom”) (baen.com). Talia’s war memories and phantom limb sensations immediately establish her as a capable but scarred individual, longing for redemption and peace. Her internal conflict – soldier versus civilian – is already in focus.

It’s been my longstanding quip that the neural implant is separate from the phantom, but this is a nuance that can only truly be understood by reading the book, which it obviously didn’t as you’ll see below.

ChatGPT: Worldbuilding Details: The setting is a colony world a few hundred years in the future, after Earth’s great powers fractured. Gōruden attracts those seeking a simpler life “less encumbered by government” control (paulsemel.com). We learn that high technology is scarce outside the capital; Tsuri and other frontier areas deliberately maintain an Old West meets Meiji-era Japan aesthetic (paulsemel.com). This means wood-built towns, steam-era machinery, and maybe even sword-carrying traditions, contrasting with Earth’s advanced but oppressive society. We also learn Talia’s trip to Gōruden took a long time (about a month), emphasizing the planet’s remoteness.

And here we have the first AI-ism in that last sentence. Talia wasn’t traveling to Gōruden, the planet. She had been on Gōruden for years and was traveling from Sakura (the planetary capital) to Tsurui, and that’s the trip that takes about a month. Remember me telling you that LLMs conflate locations, people, and sequencing? This is a perfect example of that. Just 266 words in, and we have our first AI-ism.

ChatGPT: Themes: Freedom vs. Control is immediately apparent. Talia’s journey represents personal freedom – she literally bought her freedom from indenture to escape the controlled life under corporate/Earth oversight (baen.com). The individualism vs. collectivism theme also surfaces: Talia is striking out alone to carve her own path, away from the military machine that once defined her. Additionally, tradition vs. progress appears in the choice of a low-tech colony over high-tech Earth; Talia (and other colonists) prefer a humble, old-fashioned lifestyle if it means personal liberty (paulsemel.com). The chapter sets up Talia’s hope that on Gōruden she can be an individual, not a weapon, implicitly asking whether she can truly escape her violent “old calling.”

So far so good on the themes (they are extensively discussed in online resources) but there really is no mention on what’s actually in chapter one. Basically, Talia is caught naked in an onsen bathing pool by her old sniper partner, Lyle, who holds her at gunpoint. Lyle believes rumors that Signore Contesti hired an assassin to kill him and suspects Talia might be that assassin.

Instead of what’s in the actual chapter, we get generated content on what ChatGPT gathered and rephrased from Paul Semel’s interview and Baen’s ad-copy. I’m not even sure it bothered to read the free chapters that are up on Baen’s website.

On to “Chapter 2.”

ChatGPT: Chapter 2: Reunion at Sakura Spaceport

Major Plot Events: Talia’s journey includes a stopover in Sakura, Gōruden’s planetary capital city. Here, modern 21st-century-level technology exists – there’s a bustling spaceport, neon signs in multiple languages, and advanced amenities (paulsemel.com). After the rustic starship passage, Sakura feels cosmopolitan. Talia takes the opportunity to relax at the Full Moon Bath House, a Japanese-style public bath, to wash off the dust of travel. Unexpectedly, she reunites with an old friend, Lyle Monroe, who meets her there. Lyle is the person who alerted her to the job opening in Tsuri. During this meeting, he gives her insight into the situation awaiting her: Tsuri’s tranquility is threatened by a newcomer’s schemes. This foreshadows the conflict to come.

Talia’s journey includes what now? She’s just left Sakura and hasn’t returned yet. What rustic starship passage? Lyle most definitely didn’t alert her to the job opening. ChatGPT didn’t read any of Chapter 2. It just made up some stuff. Talia did start out in the Full Moon Onsen, in Chapter 1, and yes her reunion with Lyle was unexpected, but this is a total misrepresentation of what happened.

Hallucinations

So just a few hundred words in, we see how the LLM generated, as it made up, its own damned story built loosely on my elements and ideas.

ChatGPT: Character Introductions & Development: Lyle Monroe is introduced as Talia’s former comrade and close friend. They bonded “in blood via war,” much like old gunslinger friends reuniting (paulsemel.com). Lyle came to Gōruden earlier and now serves as sheriff of Tsuri, trying to keep the peace. However, Talia immediately notices a change in him – Lyle appears worn-down and uneasy, with hints of a drinking habit. His reunion with Talia is warm but strained; through subtext we see he’s grappling with personal demons and the deteriorating state of Tsuri. Lyle’s vulnerability and his reliance on Talia’s help start to build his arc of redemption and dependence on friendship. We also glimpse Talia’s softer side here: she genuinely cares for her friend and feels a sense of duty to help him.

Wait now. According to ChatGPT, Lyle has a drinking habit. I certainly didn’t write that, because he doesn’t, not at the start of the story. Anyone who has actually read the book would know what Lyle’s actual problem was. Nothing is mentioned of what actually happens in Chapter 2, just so you know.

ChatGPT: Worldbuilding Details: Sakura’s brief depiction contrasts with Tsuri’s expected simplicity. In Sakura, we see late-21st-century tech: efficient public transit, powered infrastructure, and the presence of corporate offices – a reminder that Earth-based corporations still operate herepaulsemel.com. The Full Moon Bath House itself reflects Gōruden’s unique cultural blend: a traditional Japanese bath in a space-age city, symbolizing how settlers imported pieces of Earth’s cultures. We also learn travel on Gōruden can be slow and arduous (a month of travel, partly due to light-speed limits and the planet’s frontier nature), reinforcing how far from Earth’s reach this colony is (upstreamreviews.substack.com). The political backdrop is hinted: Earth’s nations have been replaced by corporate and oligarchic powers, and Gōruden’s settlers ultimately aim to declare independence from Earth once they solve the mystery of why the first attempt to colonize Gōruden failed (paulsemel.com). This hint about a first colonization failure and the need to prevent it lays groundwork for the terraforming secret plot.

For the record, I’d never write “efficient public transit” into anything. Or any “efficient public” anything. (If I do, it’s my secret signal to you that I’ve been kidnapped and to send in a rescue team.) The Full Moon Onsen is in Tsurui (a remote village on the island of Tatarka), not in Sakura. And while travel on Gōruden, is slow and arduous, it’s not because of “light-speed limits.” It doesn’t understand that on-planet travel is not light-speed dependent. It just associates travel in a scifi milieu with “light-speed” and lumps it in with travel-related verbiage because LLMs are blooming idiots.

ChatGPT: Themes: Loyalty and duty emerge strongly. Talia intended to avoid entanglements, but her loyalty to Lyle – a friend who once bled with her in war – draws her in, echoing the theme that “there is no greater love than to lay down one’s life for one’s friends”paulsemel.com. This ties into the individual vs. collectivism theme: Talia’s personal quest for peace is immediately challenged by her social bonds and obligations to others. Freedom vs. Control also appears in conversation – Lyle implies an outside force (the “newcomer” in Tsuri) is trying to impose control, threatening the freedom of the frontier folk. The contrast of tradition vs. progress is visible in Sakura vs. Tsuri: Talia appreciates the conveniences of Sakura but is eager to head to Tsuri’s simpler life, highlighting the colonists’ conscious choice to forsake some progress for autonomypaulsemel.com.

While these themes are there, they aren’t the crux of Chapter 2, not by any measure. It’s just paraphrasing the Semel interview.

As the “analysis” progresses, the LLM continues to make stuff up and I will highlight some of its “hallucinations” for you.

ChatGPT: Talia and Lyle travel from Sakura to Tsuri.

Nope. Logan and Talia travel from Sakura to Tsurui.

Chat GPT: They board a mag-lev train

News to me. Pretty sure I didn’t put in a mag-lev train anywhere. Some of the corporate types were thinking of developing Gōruden with such tech, but that’s it.

ChatGPT: Near the end of the chapter, an incident on the train ride tests Talia’s resolve to stay low-profile: a mechanical failure causes a near derailment, and Talia uses her sharp reflexes (enhanced by her military implant) to help avert disaster. Though she saves lives quietly, a few passengers remark on her uncanny calm and precision, hinting at her extraordinary skill.

Nope. Never happened. There was no derailment and Talia certainly couldn’t have prevented it being a passenger. Talia is not a know-it-all, do-it-all Mary-Sue. You know what has train derailments, especially ones averted by the story’s hero? Westerns, that’s what. The LLM took what it “knew” about Western plot lines and just plugged it in there. It basically took the “brain drippings” of The Consensus (TM) and ran with it because the consensus (including its slobbering worship of “agency”) rules when it comes to generated answers based on statistics/probabilities.

Some highlights:

ChatGPT: There’s mention of an abandoned old settlement site and a terraforming facility out in the wilderness that was never fully operational – a clue to the planet’s secrets.

No there’s not.

ChatGPT: The townsfolk initially greet Talia with warmth and curiosity – an outsider is big news in a small town.

No they don’t.

ChatGPT: one of Contesti’s hirelings, a stern man in a corporate security uniform, is arguing with a local farmer about a piece of land.

What now? News to me.

ChatGPT: Jerrod notices Talia too; he gives her a curt nod on the street, a gesture that feels more probing than polite.

Nope. Not even close. Where did this come from? Other Westerns. It continues with this farmer-overlord dispute, basically just making up more stuff as it goes. Ridiculous.

ChatGPT: In a pivotal moment, Talia shoots a series of distant targets with uncanny accuracy, impressing Contesti and Jerrod Rhodes (who observes silently from the sidelines). Contesti hires her on the spot as a marksmanship instructor for the town.

I’m pretty sure this happened in some other Western or is a pretty typical plot point in a plain old Western.

ChatGPT: He’s introduced now in person as Contesti’s right-hand man, a former colonel. Jerrod is polite but cold toward Talia; he even tests her by unexpectedly tossing a small drone target into the air during her shooting trial. Talia nails the drone with one shot, and Jerrod gives a thin smile.

Nope. Nothing like this ever happened, not in my story. In someone else’s maybe. It’s using it’s “knowledge” of what typically happens in a Western, that’s all. And by the way, that is exactly what it’s going to do when you ask it “write” your story for you—generic stuff as written by everyone else. No wonder readers aren’t reading and don’t want to read AI-generated stories.

At this point the LLM is just telling its own story, weaving in some elements from mine, willy nilly, with many elements not making sense for the world I built.

ChatGPT:

His backstory with the redhead (named Siobhan, we learn) exposes his tragic flaw: he’s a decent man who was outmaneuvered by his own heart.

She sees modest wood and paper buildings influenced by Japanese architecture (curved eaves, a temple-like town hall) mixed with classic Old West elements like a saloon and hitching posts for mechanical horses.

During the class, Talia forms a rapport with a bright teenage girl named Emiko (one of the few true locals in the group).

We see more of everyday life and dangers on Gōruden. The raug (native predator) incident highlights that beyond human conflicts, the frontier has its own perils. It also underscores why marksmanship is valued here.

Major Plot Events: In the final chapter, the immediate threat of the combat drones is resolved in a clever, collective way. Under Talia and Lyle’s direction, the townsfolk use their frontier ingenuity to disable or destroy the incoming drones – they herd them into a choke point between buildings and use an old EMP device (salvaged from the first colony’s equipment cache, as hinted by Dr. Kamakura’s notes) to knock them out.

It just keeps going, getting more generative as it goes.

If you want read ChatGPT’s entire hallucination, it’s here. If you want to read the actual story, it can be bought here.

Conclusion

If it’s not obvious to you at this point that the LLM didn’t “read” anything, but simply took some material from the ad-copy and reviews and then just made up its own story based on what was statistically based on Westerns, then I don’t know what to say or how to clarify the generative nature of generative-AI for you. In order for any of this to be usable, you’d have be quite familiar with the story to know that you can ignore pretty much everything after the Chapter 1 header. About the only facts here occur in the first two sentences of the Summary.

The LLM made up a 10,000-word story it pieced together from known Western plot devices, tropes, etc. And it certainly wasn’t my story.

Does this remind you of the the child who does the absolute minimum to pass, who has no interest in actually learning anything, just passing a test, or the minimum-effort employee who relies on likability or personality to keep his job, the one we all watch skate through the workday while we do the real work? It certainly does me.