Recently I've been working on a RAG, understanding how it works, how RAGs and Agentic technology produces tokens, and how they are trained to recognize material, and how the various layering of training accumulates information in the multi-layer massive neural network that gets trained. As a result of that, and my NLP background, I now have a theory about em-dashes and long sentences.
When an AI trains on text, it tokenizes the content. The tokens it uses are typically byte-pair encodings (BPE) that are learned over a corpus; these become a new "alphabet" for language learned from character frequencies and adjacencies. The alphabet produced can be defined to be a fixed size or achieve a certain encoding metric over text. This results in a token alphabet of say 30,000 mappings. Base letters and punctuation and numbers get a BPE encoding that is an identity mapping. If the input language is expressed in the Unicode character set, and the text is UTF-8 encoded, then each byte is either a UTF-8 lower 128 character, or a lead byte for longer encodings of unicode text. BPE works on the bytes. Thus, any sequence of UTF-8 encoded text can be represented in the initial 256 byte pair codes that are identity mapped to bytes 0 - 255. Thereafter, the first most frequent pairing gets encoded as character 256. The text is rencoded to the new langauge left to right using the new "character", and the process repeats. As a result of this process, commonly used upper 128 characters in ANSI, such as curly-quotes, mathematical symbols, en-dash, em-dash, elipses (...), et cetera, might get encoded in a single token. Sadly, emoji such as 🎉might require 4 tokens to encode.
That's the first stage. Then the neural network training begins. As that training continues, it will start to collapse concept in language that represent certain features. One of the features of language is the "pause". The period, comma, colon, semi-colon, en-dash, and em-dash all represent pauses, and () pairs also invoke the "pause" mechanics. Ending of a concept or thought. The period plays a dual role as ending of a thought or concept but also plays a separate role (as does comma) of a digit separator and/or decimal separator, as well as an initial separator. The LLM learns the idea of a conversational pause. Anywhere a period, comma, colon, semi-colon, en-dash or parenthetical expression appear, the em-dash can step in without further loss of meaning -- making it the universal concept representing a pause. The LLM learns this and then decides that any pause deserves its own cluster in the model as it were. So all of these pauses lean towards the pause cluster. Since -- can fit in both places the model might cluster -- with , and -- with ; and so forth, and when these clusters in the model again become bigger clusters of meaning, the representation of them, reversed out becomes the token having the biggest hold on the space, the --, rather than the more meaningful siblings.
I believe this is an example of learned bias. But it also means something else. Thought ending behavior the period, conflates with em-dash, which is also concept ending behavior, indistinguishable from thought ending. And thereafter, the training loses track of the human concept of the need for a bigger pause (the period) to make concepts easier to understand.
As a result of this, the training leans towards longer sentences, some taking three lines or more to complete, and therefore, you get lost in the AI noise -- causing you to scratch you head and wonder "is this ever going to stop?" and then finally, it does -- or not, based on the training process and random noise inputs that keep the output variable and interesting
To combat this, I've devised a set if writing instructions (and skills because most agents I work with lose track of their instructions after a few context compressions where they haven't been important). It basically says replace em-dashes with other forms of punctuation, period, comma, colon, semi-colon, or parentheses for conversational asides and clarifications. I have another set of skills based on HHS simple language training materials that breaks sentences down to less than N words depending on the reading score I'm reaching for. For technical content, about 25 words is good, 20 is better for conversational text.
As I said, this is a theory, based on (what little) I know about how this stuff works, but it might be worth investigating further. The bigger pause (the blank line or newline) likely gets different treatment during learning, which is why agents might be keen on writing shorter paragraphs. As a theory, it's worth investigating further, but I'm not at a point where I would be able to do that myself.
0 comments:
Post a Comment