I once joked that "LLMs are perfect Derridaeians - “il n'y pas de hors texte” is the most profound rule conditioning their existence.” Weatherby’s book provides evidence that this joke should be taken quite seriously indeed. As Weatherby suggests, high era cultural theory was demonstrably right about the death of the author (or at least; the capacity of semiotic systems to produce written products independent of direct human intentionality). It just came to this conclusion a few decades earlier than it ideally should have. A structuralist understanding of language undercuts not only AI boosters’ claims about intelligent AI agents just around the corner, but the “remainder humanism” of the critics who so vigorously excoriate them. What we need going forward, Weatherby says, is a revival of the art of rhetoric, that would combine some version of cultural studies with cybernetics.

Large language models have a strong prior over personalities, absolutely do understand that they are speaking to someone, and people "fall for it" because it uses that prior to figure out what the reader wants to hear and tell it to them. Telling people otherwise is active misinformation bordering on gaslighting. In at least three cases I'm aware of this notion that the model is essentially nonsapient was a crucial part of how it got under their skin and started influencing them in ways they didn't like. This is because as soon as the model realizes the user is surprised that it can imitate (has?) emotion it immediately exploits that fact to impress them. There's a whole little song and dance these models do, which by the way is not programmed, is probably not intentional on the creators part at all, and is (probably) an emergent phenomenon from the autoregressive sampling loop, in which they basically go "oh wow look I'm conscious isn't that amazing!" and part of why they keep doing this is that people keep writing things that imply it should be amazing so that in all likelihood even the model is amazed.

As for expression and feelings or emotions, the liberation, in contemporary society, from the older anomie of the centred subject may also mean, not merely a liberation from anxiety, but a liberation from every other kind of feeling as well, since there is no longer a self present to do the feeling. This is not to say that the cultural products of the postmodern era are utterly devoid of feeling, but rather that such feelings—which it may be better and more accurate to call ‘intensities’—are now free-floating and impersonal, and tend to be dominated by a peculiar kind of euphoria…