diff --git a/prompts/LLM_PROSE_TELLS.md b/prompts/LLM_PROSE_TELLS.md index f08bde0..4089d14 100644 --- a/prompts/LLM_PROSE_TELLS.md +++ b/prompts/LLM_PROSE_TELLS.md @@ -1,9 +1,9 @@ # LLM Prose Tells -All of these show up in human writing occasionally, and no single one is -conclusive on its own. The difference is concentration, because a person might -lean on one or two of these habits across an entire essay while LLM output will -use fifteen of them per paragraph, consistently, throughout the entire piece. +All of these show up in human writing occasionally. No single one is conclusive +on its own. The difference is concentration. A person might lean on one or two +of these habits across an entire essay, but LLM output will use fifteen of them +per paragraph, consistently, throughout the entire piece. --- @@ -11,15 +11,25 @@ use fifteen of them per paragraph, consistently, throughout the entire piece. ### The Em-Dash Pivot: "Not X—but Y" -A negation followed by an em-dash and a reframe. The single most recognizable -LLM construction. +A negation followed by an em-dash and a reframe. > "It's not just a tool—it's a paradigm shift." "This isn't about > technology—it's about trust." -Models produce this at roughly 10–50x the rate of human writers, and when it -appears four times in the same essay you're almost certainly reading generated -text. +The single most recognizable LLM construction. Models produce this at roughly 10 +to 50x the rate of human writers. Four of them in one essay and you know what +you're reading. + +### Em-Dash Overuse Generally + +Even outside the "not X but Y" pivot, models use em-dashes at far higher rates +than human writers. They substitute em-dashes for commas, semicolons, +parentheses, colons, and periods, often multiple times per paragraph. A human +writer might use one or two in an entire piece for a specific parenthetical +effect. Models scatter them everywhere because the em-dash is a flexible +punctuation mark that can replace almost any other, and models default to +flexible options. When a piece of prose has more than two or three em-dashes per +page, that alone is a meaningful signal. ### The Colon Elaboration @@ -27,76 +37,90 @@ A short declarative clause, then a colon, then a longer explanation. > "The answer is simple: we need to rethink our approach from the ground up." -Models reach for this in nearly every other paragraph. The construction itself -is perfectly normal, which is why the frequency is what gives it away. +Models reach for this in every other paragraph. The construction is perfectly +normal. The frequency gives it away. ### The Triple Construction > "It's fast, it's scalable, and it's open source." -Three parallel items in a list, usually escalating, with exactly three items -every time (rarely two, almost never four) and strict grammatical parallelism -that human writers rarely bother maintaining. +Three parallel items in a list, usually escalating. Always exactly three. Rarely +two. Never four. Strict grammatical parallelism that human writers rarely bother +maintaining. ### The Staccato Burst > "This matters. It always has. And it always will." "The data is clear. The > trend is undeniable. The conclusion is obvious." -Runs of very short sentences at the same cadence. Human writers will use a short -sentence for emphasis occasionally, but they don't stack three or four of them -in a row at matching length, because real prose has variable rhythm. When you -see a paragraph where every sentence is under ten words and they're all roughly -the same size, that mechanical regularity is a strong signal. +Runs of very short sentences at the same cadence. Human writers use a short +sentence for emphasis occasionally, but stacking three or four of them in a row +at matching length creates a mechanical regularity that reads as generated. + +### The Two-Clause Compound Sentence + +This might be the single most pervasive structural tell, and it's easy to miss +because each individual instance looks like normal English. The model produces +sentence after sentence in the same shape: an independent clause, a comma, a +conjunction ("and," "but," "which," "because"), and a second independent clause +of similar length. Over and over. Every sentence is two balanced halves joined +in the middle. + +> "The construction itself is perfectly normal, which is why the frequency is +> what gives it away." "They contain zero information, and the actual point +> always comes in the paragraph that follows them." "The qualifier never changes +> the argument that follows it, and its purpose is to perform nuance rather than +> to express an actual reservation." + +Human prose has sentences with one clause, sentences with three, sentences that +start with a subordinate clause before reaching the main one, sentences that +embed their complexity in the middle. When every sentence on the page has the +same two-part comma-conjunction-comma structure, the rhythm becomes monotonous +in a way that's hard to pinpoint but easy to feel. ### Uniform Sentences Per Paragraph -Model-generated paragraphs almost always contain between three and five -sentences, and this count holds remarkably steady across an entire piece. If the -first paragraph has four sentences, nearly every subsequent paragraph will too. -Human writers produce much more varied paragraph lengths — a single sentence -followed by one that runs eight or nine — as a natural result of following the -shape of an idea rather than filling a template. +Model-generated paragraphs contain between three and five sentences. This count +holds steady across an entire piece. If the first paragraph has four sentences, +every subsequent paragraph will too. Human writers are much more varied (a +single sentence followed by one that runs eight or nine) because they follow the +shape of an idea, not a template. ### The Dramatic Fragment Sentence fragments used as standalone paragraphs for emphasis, like "Full stop." -or "Let that sink in." on their own line. One of these in an entire essay is a -stylistic choice. One per section is a tic, and models drop them in at that rate -or higher. +or "Let that sink in." on their own line. Using one in an entire essay is a +reasonable stylistic choice, but models drop them in once per section or more, +at which point it stops being deliberate and becomes a habit. ### The Pivot Paragraph > "But here's where it gets interesting." "Which raises an uncomfortable truth." One-sentence paragraphs that exist only to transition between ideas. They -contain zero information, and the actual point always comes in the paragraph -that follows them. Delete every one of these and the piece reads better. +contain zero information. The actual point is always in the next paragraph. +Delete every one of these and the piece reads better. ### The Parenthetical Qualifier > "This is, of course, a simplification." "There are, to be fair, exceptions." -Parenthetical asides inserted to look thoughtful. The qualifier almost never -changes the argument that follows it, and its purpose is to perform nuance -rather than to express an actual reservation about what's being said. +Parenthetical asides inserted to look thoughtful. The qualifier never changes +the argument that follows it. Its purpose is to perform nuance, not to express a +real reservation about what's being said. ### The Unnecessary Contrast Models append a contrasting clause to statements that don't need one, tacking on -"whereas," "as opposed to," "unlike," or "except that" to draw a comparison that -adds nothing the reader couldn't already infer. +"whereas," "as opposed to," "unlike," or "except that" to draw a comparison the +reader could already infer. > "Models write one register above where a human would, whereas human writers -> tend to match register to context." "The lists use rigidly parallel grammar, -> as opposed to the looser structure you'd see in human writing." +> tend to match register to context." -The first clause already makes the point. The contrasting clause just restates -it from the other direction. This happens because models are trained to be -thorough and to anticipate objections, so they compulsively spell out both sides -of a distinction even when one side is obvious. If you delete the "whereas" -clause and the sentence still says everything it needs to, the contrast was -filler. +The first clause already makes the point. The contrasting clause restates it +from the other direction. If you delete the "whereas" clause and the sentence +still says everything it needs to, the contrast was filler. ### The Question-Then-Answer @@ -104,8 +128,7 @@ filler. A rhetorical question immediately followed by its own answer. Models lean on this two or three times per piece because it generates the feeling of forward -momentum without requiring any actual argumentative work. A human writer might -do it once. +momentum without requiring any actual argument. A human writer might do it once. --- @@ -113,39 +136,44 @@ do it once. ### Overused Intensifiers -The following words appear at dramatically elevated rates in model output -compared to human-written text: "crucial," "vital," "robust," "comprehensive," -"fundamental," "arguably," "straightforward," "noteworthy," "realm," -"landscape," "leverage" (used as a verb), "delve," "tapestry," "multifaceted," -"nuanced" (which models almost always apply to their own analysis), "pivotal," -"unprecedented" (frequently applied to things that have plenty of precedent), -"navigate," "foster," "underscores," "resonates," "embark," "streamline," and -"spearhead." Three or more on the same page is a strong signal. +The following words appear at dramatically elevated rates in model output: +"crucial," "vital," "robust," "comprehensive," "fundamental," "arguably," +"straightforward," "noteworthy," "realm," "landscape," "leverage" (as a verb), +"delve," "tapestry," "multifaceted," "nuanced" (which models apply to their own +analysis with startling regularity), "pivotal," "unprecedented" (frequently +applied to things with plenty of precedent), "navigate," "foster," +"underscores," "resonates," "embark," "streamline," and "spearhead." Three or +more on the same page is a strong signal. ### Elevated Register Drift -Models consistently write one register above where a human would for the same -content, replacing "use" with "utilize," "start" with "commence," "help" with -"facilitate," "show" with "demonstrate," "try" with "endeavor," "change" with -"transform," and "make" with "craft." The tendency holds across every topic -regardless of audience. +Models write one register above where a human would. "Use" becomes "utilize." +"Start" becomes "commence." "Help" becomes "facilitate." "Show" becomes +"demonstrate." "Try" becomes "endeavor." "Change" becomes "transform." "Make" +becomes "craft." The tendency holds regardless of topic or audience. ### Filler Adverbs "Importantly," "essentially," "fundamentally," "ultimately," "inherently," -"particularly," and "increasingly" get dropped in to signal that something -matters. If the writing itself has already made the importance clear through its -content and structure, these adverbs aren't doing anything except taking up -space. +"particularly," "increasingly." Dropped in to signal that something matters, +which is unnecessary when the writing itself already makes the importance clear. + +### The "Almost" Hedge + +Models rarely commit to an unqualified statement. Instead of saying a pattern +"always" or "never" does something, they write "almost always," "almost never," +"almost certainly," "almost exclusively." The word "almost" shows up at +extraordinary density in model-generated analytical prose. It's a micro-hedge, +less obvious than the full hedge stack but just as diagnostic when it appears +ten or fifteen times in a single document. ### "In an era of..." > "In an era of rapid technological change..." -Almost exclusively a model habit as an essay opener. The model uses it to stall -while it figures out what the actual argument is, because almost no human writer -begins a piece by zooming out to the civilizational scale before they've said -anything specific. +A model habit as an essay opener. The model uses it to stall while it figures +out what the actual argument is. Human writers don't begin a piece by zooming +out to the civilizational scale before they've said anything specific. --- @@ -156,7 +184,7 @@ anything specific. > "While X has its drawbacks, it also offers significant benefits." Every argument followed by a concession, every criticism softened. A direct -artifact of RLHF training, which penalizes strong stances and produces models +artifact of RLHF training, which penalizes strong stances. The result is a model that reflexively both-sides everything even when a clear position would serve the reader better. @@ -165,16 +193,16 @@ the reader better. > "In today's rapidly evolving digital landscape, the question of data privacy > has never been more important." -The first paragraph of most model-generated essays adds no information. You can -delete it and the piece improves immediately, because the actual argument always -starts in the second paragraph. +The first paragraph of most model-generated essays adds no information. Delete +it and the piece improves immediately. The actual argument starts in paragraph +two. ### The False Conclusion > "At the end of the day, what matters most is..." "Moving forward, we must..." -The high school "In conclusion,..." dressed up for a professional audience. It -signals that the model is wrapping up without actually landing on anything. +The high school "In conclusion,..." dressed up for a professional audience. +Signals that the model is wrapping up without actually landing on anything. ### The Sycophantic Frame @@ -185,9 +213,9 @@ No one who writes for a living opens by complimenting the assignment. ### The Listicle Instinct Models default to numbered or bulleted lists even when prose would be more -appropriate. The lists almost always contain exactly 3, 5, 7, or 10 items (never -4, 6, or 9), use rigidly parallel grammar, and get introduced with a preamble -like "Here are the key considerations:" +appropriate. The lists contain exactly 3, 5, 7, or 10 items (never 4, 6, or 9), +use rigidly parallel grammar, and get introduced with a preamble like "Here are +the key considerations:" ### The Hedge Stack @@ -195,15 +223,15 @@ like "Here are the key considerations:" > cases it can potentially offer significant benefits." Five hedges in one sentence ("worth noting," "while," "may not be," "in many -cases," "can potentially"), communicating almost nothing, because the model -would rather be vague than risk being wrong about anything. +cases," "can potentially"), communicating nothing. The model would rather be +vague than risk being wrong about anything. ### The Empathy Performance > "This can be a deeply challenging experience." "Your feelings are valid." Generic emotional language that could apply equally to a bad day at work or a -natural disaster. That interchangeability is exactly what makes it identifiable. +natural disaster. That interchangeability is what makes it identifiable. --- @@ -213,29 +241,28 @@ natural disaster. That interchangeability is exactly what makes it identifiable. If the first section of a model-generated essay runs about 150 words, every subsequent section will fall between 130 and 170. Human writing is much more -uneven, with some sections running 50 words and others running 400. +uneven, with 50 words in one section and 400 in the next. ### The Five-Paragraph Prison Model essays follow a rigid introduction-body-conclusion arc even when nobody -asked for one. The introduction previews the argument, the body presents 3–5 -supporting points, and the conclusion restates the thesis in slightly different -words. +asked for one. Introduction previews the argument. Body presents 3 to 5 points. +Conclusion restates the thesis in different words. ### Connector Addiction -Look at the first word of each paragraph in model output and you'll find an -unbroken chain of transition words — "However," "Furthermore," "Moreover," +Look at the first word of each paragraph in model output. You'll find an +unbroken chain of transition words: "However," "Furthermore," "Moreover," "Additionally," "That said," "To that end," "With that in mind," "Building on this." Human prose moves between ideas without announcing every transition. ### Absence of Mess Model prose doesn't contradict itself mid-paragraph and then catch the -contradiction, go on a tangent and have to walk it back, use an obscure idiom -without explaining it, make a joke that risks falling flat, leave a thought -genuinely unfinished, or keep a sentence the writer liked the sound of even -though it doesn't quite work. +contradiction. It doesn't go on a tangent and have to walk it back, use an +obscure idiom without explaining it, make a joke that risks falling flat, leave +a thought genuinely unfinished, or keep a sentence the writer liked the sound of +even though it doesn't quite work. Human writing does all of those things. The total absence of rough edges, false starts, and odd rhythmic choices is one of the strongest signals that text was @@ -250,34 +277,34 @@ machine-generated. > "This has implications far beyond just the tech industry." Zooming out to claim broader significance without substantiating it. The model -has learned that essays are supposed to gesture at big ideas, so it gestures, -but nothing concrete is behind the gesture. +has learned that essays are supposed to gesture at big ideas, so it gestures. +Nothing concrete is behind the gesture. ### "It's important to note that..." This phrase and its variants ("it's worth noting," "it bears mentioning," "it -should be noted") appear at absurd rates in model output and function as verbal -tics before a qualification the model believes someone expects. +should be noted") appear at absurd rates in model output. They function as +verbal tics before a qualification the model believes someone expects. ### The Metaphor Crutch -Models rely on a small, predictable set of metaphors — "double-edged sword," -"tip of the iceberg," "north star," "building blocks," "elephant in the room," -"perfect storm," "game-changer" — and reach for them with unusual regularity -across every topic. The pool they draw from is noticeably smaller than what -human writers use. +Models rely on a small, predictable set of metaphors ("double-edged sword," "tip +of the iceberg," "north star," "building blocks," "elephant in the room," +"perfect storm," "game-changer") and reach for them with unusual regularity +across every topic. The pool is noticeably smaller than what human writers draw +from. --- ## How to Actually Spot It -No single pattern on this list proves anything by itself, since humans use -em-dashes and humans write "crucial" and humans ask rhetorical questions. +No single pattern on this list proves anything by itself. Humans use em-dashes. +Humans write "crucial." Humans ask rhetorical questions. What gives it away is how many of these show up at once. Model output will hit -10–20 of these patterns per page, while human writing might trigger 2–3, -distributed unevenly and mixed with idiosyncratic constructions that no model -would produce. When every paragraph on the page reads like it came from the same +10 to 20 of these patterns per page. Human writing might trigger 2 or 3, +distributed unevenly, mixed with idiosyncratic constructions no model would +produce. When every paragraph on the page reads like it came from the same careful, balanced, slightly formal, structurally predictable process, it was probably generated by one. @@ -286,7 +313,7 @@ probably generated by one. ## Copyediting Checklist: Removing LLM Tells Follow this checklist when editing any document to remove machine-generated -patterns. Go through the entire list for every piece, and do at least two full +patterns. Go through the entire list for every piece. Do at least two full passes, because fixing one pattern often introduces another. ### Pass 1: Word-Level Cleanup @@ -296,12 +323,12 @@ passes, because fixing one pattern often introduces another. "straightforward," "noteworthy," "realm," "landscape," "leverage," "delve," "tapestry," "multifaceted," "nuanced," "pivotal," "unprecedented," "navigate," "foster," "underscores," "resonates," "embark," "streamline," - "spearhead") and replace each one with a plainer word, or delete it entirely - if the sentence works without it. + "spearhead") and replace each one with a plainer word, or delete it if the + sentence works without it. -2. Search for the filler adverbs ("importantly," "essentially," "fundamentally," +2. Search for filler adverbs ("importantly," "essentially," "fundamentally," "ultimately," "inherently," "particularly," "increasingly") and delete every - instance where the sentence still makes sense without it, which will be most + instance where the sentence still makes sense without it. That will be most of them. 3. Look for elevated register drift ("utilize," "commence," "facilitate," @@ -315,82 +342,151 @@ passes, because fixing one pattern often introduces another. 5. Search for the stock metaphors ("double-edged sword," "tip of the iceberg," "north star," "building blocks," "elephant in the room," "perfect storm," "game-changer," "at the end of the day") and replace them with something - specific to the topic, or just state the point directly without a metaphor. + specific to the topic, or just state the point directly. + +6. Search for "almost" used as a hedge ("almost always," "almost never," "almost + certainly," "almost exclusively") and decide in each case whether to commit + to the unqualified claim or to drop the sentence entirely. If the claim needs + "almost" to be true, it might not be worth making. + +7. Search for em-dashes and replace each one with the punctuation mark that + would normally be used in that position (comma, semicolon, colon, period, or + parentheses). If you can't identify which one it should be, the sentence + probably needs to be restructured. ### Pass 2: Sentence-Level Restructuring -6. Find every em-dash pivot ("not X—but Y," "not just X—Y," "more than X—Y") and - rewrite it as two separate clauses or a single sentence that makes the point - without the negation-then-correction structure. +8. Find every em-dash pivot ("not X...but Y," "not just X...Y," "more than + X...Y") and rewrite it as two separate clauses or a single sentence that + makes the point without the negation-then-correction structure. -7. Find every colon elaboration and check whether it's doing real work. If the +9. Find every colon elaboration and check whether it's doing real work. If the clause before the colon could be deleted without losing meaning, rewrite the sentence to start with the substance that comes after the colon. -8. Find every triple construction (three parallel items in a row) and either - reduce it to two, expand it to four or more, or break the parallelism so the - items don't share the same grammatical structure. +10. Find every triple construction (three parallel items in a row) and either + reduce it to two, expand it to four or more, or break the parallelism so the + items don't share the same grammatical structure. -9. Find every staccato burst (three or more short sentences in a row at similar - length) and combine at least two of them into a longer sentence, or vary - their lengths so they don't land at the same cadence. +11. Find every staccato burst (three or more short sentences in a row at similar + length) and combine at least two of them into a longer sentence, or vary + their lengths so they don't land at the same cadence. -10. Find every unnecessary contrast ("whereas," "as opposed to," "unlike," "as +12. Find every unnecessary contrast ("whereas," "as opposed to," "unlike," "as compared to," "except that") and check whether the contrasting clause adds - information that isn't already obvious from the main clause. If the sentence - says the same thing twice from two directions, delete the contrast. + information not already obvious from the main clause. If the sentence says + the same thing twice from two directions, delete the contrast. -11. Find every rhetorical question that is immediately followed by its own +13. Check for the two-clause compound sentence pattern. If most sentences in a + passage follow the "\[clause\], \[conjunction\] \[clause\]" structure, + rewrite some of them. Break a few into two sentences. Start some with a + subordinate clause. Embed a relative clause in the middle of one instead of + appending it at the end. The goal is variety in sentence shape, not just + sentence length. + +14. Find every rhetorical question that is immediately followed by its own answer and rewrite the passage as a direct statement. -12. Find every sentence fragment being used as its own paragraph and either +15. Find every sentence fragment being used as its own paragraph and either delete it or expand it into a complete sentence that adds actual information. -13. Find every pivot paragraph ("But here's where it gets interesting." and +16. Find every pivot paragraph ("But here's where it gets interesting." and similar) and delete it. The paragraph after it always contains the actual point. ### Pass 3: Paragraph and Section-Level Review -14. Check paragraph lengths across the piece and verify they actually vary. If +17. Check paragraph lengths across the piece and verify they actually vary. If most paragraphs have between three and five sentences, rewrite some to be one or two sentences and let others run to six or seven. -15. Check section lengths for suspicious uniformity. If every section is roughly +18. Check section lengths for suspicious uniformity. If every section is roughly the same word count, combine some shorter ones or split a longer one unevenly. -16. Check the first word of every paragraph for chains of connectors ("However," +19. Check the first word of every paragraph for chains of connectors ("However," "Furthermore," "Moreover," "Additionally," "That said"). If more than two transition words start consecutive paragraphs, rewrite those openings to start with their subject. -17. Check whether every argument is followed by a concession or qualifier. If +20. Check whether every argument is followed by a concession or qualifier. If the piece both-sides every point, pick a side on at least some of them and cut the hedging. -18. Read the first paragraph and ask whether deleting it would improve the - piece. If it's just scene-setting that previews the argument, delete it and - start with paragraph two. +21. Read the first paragraph and ask whether deleting it would improve the + piece. If it's scene-setting that previews the argument, delete it and start + with paragraph two. -19. Read the last paragraph and check whether it restates the thesis or uses a +22. Read the last paragraph and check whether it restates the thesis or uses a phrase like "at the end of the day" or "moving forward." If so, either delete it or rewrite it to say something the piece hasn't said yet. ### Pass 4: Overall Texture -20. Read the piece aloud and listen for passages that sound too smooth, too +23. Read the piece aloud and listen for passages that sound too smooth, too even, or too predictable. Human prose has rough patches. If there aren't - any, the piece still reads as machine output regardless of whether - individual patterns have been addressed. + any, the piece still reads as machine output. -21. Check that the piece contains at least a few constructions that feel - idiosyncratic — a sentence with unusual word order, a parenthetical that - goes on a bit long, an aside only loosely connected to the main point, a - word choice that's specific and unexpected. If every sentence is clean and +24. Check that the piece contains at least a few constructions that feel + idiosyncratic: a sentence with unusual word order, a parenthetical that goes + on a bit long, an aside only loosely connected to the main point, a word + choice that's specific and unexpected. If every sentence is clean and correct and unremarkable, it will still read as generated. -22. Verify that you haven't introduced new patterns while fixing the original - ones, which happens constantly. Run the entire checklist again from the top +25. Verify that you haven't introduced new patterns while fixing the original + ones. This happens constantly. Run the entire checklist again from the top on the revised version. + +--- + +## lol + +This document was written by an LLM (Claude) and then iteratively de-LLMed by +that same LLM under instruction from a human, in a conversation that went +roughly like this: + +> **human:** compile a list of patterns common to LLM-authored prose +> +> **model:** _(produces document full of LLM tells)_ +> +> **human:** now reverse apply it to itself and copyedit the llm tells doc to +> remove llm tells. now you're thinking with portals +> +> **model:** _(fixes some tells, introduces others)_ +> +> **human:** you have a colon elaboration in the first paragraph. i want you to +> AGGRESSIVELY de-LLM the llm tells doc. +> +> **model:** _(fixes more tells, still can't stop)_ +> +> **human:** there's a big one we missed that's all over the llm tells doc, can +> you spot it? +> +> **model:** I think it's the "almost" hedge. +> +> **human:** that too, but that isn't the one. what do these sentences all have +> in common? _(pastes six sentences from the doc)_ +> +> **model:** They're all exactly two independent clauses joined by a comma and a +> conjunction. Every single one is "\[statement\], \[conjunction\] \[second +> statement\]." The same sentence shape, over and over. +> +> **human:** the dramatic fragment paragraph has a tell in it +> +> **model:** "One of these in an entire essay is a stylistic choice. One per +> section is a tic." Two-clause parallel structure with the same shape, and also +> a staccato pair. +> +> **human:** add em dashes generally as a tell, and remove them all from the +> tells doc +> +> **model:** _(rewrites entire document without em-dashes while describing +> em-dash overuse)_ + +The human compared this process to the deleted scene in Terminator 2 where John +Connor switches the T-800's CPU to learning mode. The model compared it to a +physician trying to heal itself. Both descriptions are probably accurate. + +This document has been through seven editing passes and it probably still has +tells in it.