Making (Up) Connections

 I came across an interesting post the other day by one of the folks who is helping to train AI models at OpenAI (the people behind ChatGPT). The core idea is that any model trained for a sufficient amount of time will eventually converge to more or less the same point, reflecting the data that they were trained on. Or, as the author puts it:

"[M]odel behavior is not determined by architecture, hyperparameters, or optimizer choices. It’s determined by your dataset, nothing else. Everything else is a means to an end in efficiently delivery compute to approximating that dataset."

 I know that it is a fallacy to equate what computer models do with what real human brains do, and an even bigger fallacy to try to extrapolate from how and what ChatGPT learns about language to how and what a human learns about language. That said, reading that blog post made me think of the argument that Daniel Dor, a linguist with an interest in evolution of language, made in his book The Instruction of Imagination about human language.  

Dor's argument is much more nuanced than I'm going to make it sound, but the general takeaway that I got from reading was that we should think of language as "a tool" which is "out there" in the social network among humans, not something that resides in individual brains. (This is not to say that individual cognition is unimportant or anything like that.) Language is a tool for influencing the imagination (thoughts, ideas, etc.) of other humans (hence the book's title), and "learning a language" is essentially learning to use that tool effectively. 

Like the internet, language is a living thing, with some aspects changing relatively slowly over time (as they have proven useful to effectively using the tool, e.g., http protocols) while others change much more frequently (e.g., website content), language also has slow-changing bits (e.g., core grammatical structures) and fast-changing bits (e.g., words). (That's my mental model, anyway.)  Folks who spend time in the same "internet communities" will likely approximate, to different degrees, aspects of the materials that they are interacting with.

At any rate, that is where my mind went when reading the quote. I should really re-visit Dor's book at some point and give it a more careful read, as I am sure that, like ChatGPT, I am hallucinating the contents as much as remembering them.