TrueAnonPod
patreon
Crackpots #46: Make Robots Behave Like Stupid Animals Again
🕑 Added 2025-09-14 15:51:15 +0000 UTCThis edition of Crackpots comes from Jayinee Basu. Jayinee is a clinical fellow in psycho-oncology and the author of the science fiction novella The City of Folding Faces. She is currently working on a sci fi novel tentatively entitled Interior Design. She is on Instagram at @yumpopink. Her writing represents her own opinions and never those of any employer or organization. She previously wrote Crackpots #16.
***
Monkeys are so sublime. How human they are! How absolutely imbued with the qualities of observation, planning, and being an asshole! Once I saw a baboon at the Prospect Park zoo wait for the perfect moment to give a good yank on his baboon fellow’s tail from below, plunging him from a rock face into the water, the both of them shrieking in what I assume to be terror and glee.
I had always thought that when robots got to the point where they’d be interacting with humans a bunch, they’d have this similar sort of monkey-ish, mischievous, lifelike-but-clearly-not-human quality about them. And for a moment it seemed like that was the direction we were heading in.
Consider:
Early videos of BigDog’s cute and ungainly attempts at locomotion, like a drunk but motivated colt
A 400 lb. egg anointed as a mall cop behaving as below:
The weird mutant squirrel picture becoming the very first piece of AI art to go viral. On my first encounter I really thought something mystical was afoot. A human couldn’t have painted it, but it was beyond my imagination that a non-human process could make something that looked so much like surreal art.
Things progressed quickly from there. I watched transfixed as a neural network dreamt live on a Twitch stream about whatever noun the chat told it to dream about (see below: knot).
(How do I even credit these? Who made them and who owns them?)
After learning to depict general objects, AI moved onto depicting with specificity over the next decade. I did pay something ludicrous like 10 bucks for an app to generate a collection of AI illustration of me on the day that everyone was doing that. It was cool that it got close to what I looked like, but frankly the real relief was that it made me hot, because what would it say about me if it had made me ugly? Despite the obvious way it smoothed out my imperfections and emphasized flattering features, it felt like it was telling the truth about how it (and therefore maybe other people) saw me. This should have set off some type of alarm in my head. The age of monkey-AI was ending and the age of sinister sycophantic AI was beginning.
Then came the LLMs. From the very beginning there was nothing monkey about them. The only times they were funny were when they made obvious errors, which of course made them less useful. Subservience and ingratiation replaced the dopey, trippy aspects of robots that I loved. Very quickly people started using them to produce social media posts, long, obnoxiously flowery comments on social media posts, blog titles, email spam, instagram captions, and basically all forms of digital text. Worst of all, at some cursed point in time ChatGPT was trained on CBT materials including the “Catch It, Check It, Change It,” technique of thought-reframing and I assume for this reason it now thinks every single response has to include the the infuriating “it’s not this, it’s that!” sentence structure. Ughhhhh.
I cannot adequately express the degree to which having to encounter this slop every day makes my eyes bleed. I recognize this is a Me problem. A lot of people (maybe even most people at this point?) don’t seem to have any trouble with this phenomenon whatsoever. There’s even some general sentiment floating around like “people always accuse good writing of being AI.” Excuse me????? Good writing?????
For example, this is a screenshot of a random public comment on a random public Facebook thread about philosophy:
The poster of this comment is pretending that he wrote this when it is obvious no human being would ever write something so stupidly florid (I double-checked by going to his profile and every single post is like this — in some of them he didn’t delete the part where the LLM addresses him by name. Whatever. This is not about him. If you’re somehow reading this, guy who posted this, I’m sure you’re a very lovely person but I beg of you to stop!!!)
It’s like the Soap Opera effect. I physically cannot tolerate it and will ask friends if we can change their TV settings when watching movies with Motion Smoothing turned on. Most friends will oblige but look at me like I’m nuts. Some will say “I used to hate it too but now I’ve gotten used to it.” And I have no doubt that this will be the case too for LLM-generated text, and maybe already is. I feel strongly that we don’t need to get used to ugly, and that there’s an axiological imperative to try to resist it.
Unlike the Soap Opera effect, the proliferation of LLM text is not a purely aesthetic issue. The ugliness of this kind of text goes beyond its ubiquity and saccharine tone. The most offensive aspect is the deception underlying an acephalic model of a language being put forward as actual language. Language is not a simple transfer of information — or rather, the information transmitted is not contained solely in its syntactical properties. It is a gesture, the weight of which lies in the context and the intention behind it. Language has powerful biological effects in those who produce and receive it, so powerful in fact that a whole field of medicine (that I practice) uses (in addition to pharmaceuticals, which is important) language as a relational modality to diagnose and treat. I acknowledge that my strong negative reaction to LLMs is based in my use of language in my profession as a psychiatrist and a writer — it’s not a bias I can or wish to get rid of as it reflects my experiences working with, and often helping, real suffering people. And in this discussion of the aesthetic value of LLMs, I feel the need to say that mimicking the aesthetics of care is dangerous when no entity actually doing the caring exists. Enter the LLM therapist.
From some Reddit threads on the topic, it seems clear that most people are aware on an intellectual level the potential risks of ChatGPT as therapist. But they give reasons why this is a better solution for them anyway than working with a human one.
First is that it is not a simple task to find a therapist. One of the great injustices of the way we’ve structured the world is that so many people have such difficulty obtaining life-saving healthcare. ChatGPT’s affirmative nature can indeed be helpful for someone on a downward spiral, just like an over-the-counter pain reliever can be helpful for someone who has injured themselves. But when chronic administration of pain relief takes the place of addressing the underlying issue, your body will let you know sooner or later that its homeostatic mechanisms are failing. But unlike the physical effects of, say, a broken leg becoming septic, the complex effects of un- or under-treated mental illness are much more difficult to identify and localize, especially for the person experiencing it as these illnesses impact the sense-making apparatus of the body. It is therefore harder to know when to seek professional help (i.e. go to the emergency room) instead of continuing to engage in a one-sided interaction that is likely making things worse. The tragic case of Adam Raine’s suicide is one such example.
Once a therapist is obtained and paid for, it is not then a simple task to engage in therapy. In a lot of people’s experience on Reddit, therapists suck and are unhelpful. Even more than the access issue above, I think this is the most problematic aspect of using LLMs for “therapy,” as it gives a false sense of what therapy actually is and should be. It seems like maybe what a lot of people are looking for out of therapy (i.e. “advice”) they would more readily find in the field of coaching, the caveat being all those that come with a completely unregulated industry. Unlike in coaching, the production and creation of the therapeutic relationship is the most important part of psychotherapy, and cannot be reproduced with a LLM.
It is true that we as doctors and therapists should suck less and be less expensive. That is true for a lot of important things (i.e. food, shelter). It’s natural to feel disappointed when you’re not getting the expected results from a service or a product that you’ve used your hard-earned money to pay for. But the therapeutic relationship is a strange product. Essentially you’re paying for a relationship and setting in which you should feel safe enough to be able to voice concerns that are really quite difficult to voice in other parts of life. Thoughts like, “I don’t think this is working” or, “I feel like I’m just talking and talking and you’re just zoning out”or “the only reason I’m showing up to these appointments is because you’re the only person of the opposite sex who will talk to me”. These critical moments of therapeutic rupture can reveal cognitive or relational patterns contributing to the larger problem at hand. Identifying a pattern opens up an opportunity to practice the skills necessary to break that pattern within the predictable confines of the therapeutic frame, so that it’s easier to identify and work with the same patterns when they inevitably occur outside of therapy in the chaotic churn of the world. Being able to voice these concerns does require substantial courage on the part of the patient, while the therapist’s task is to provide an encouraging environment. If you’ve tried to bring up concerns with your therapist multiple times and they’ve not been responsive, then it’s reasonable to find a new one. But if the same thing keeps happening with multiple therapists, then there might be something else at play besides just the quality of the therapists.
I can use myself as an example of the difficulty of “doing the work” in therapy. I sought out weekly therapy after getting some professional feedback that perhaps I was not being deferent enough in my interactions with supervisors, which confirmed my sneaking suspicion that I am and always have been an asshole. This made me feel very angry and weepy for several months. After talking about this issue in therapy for a few weeks, I felt better and decided maybe my problem wasn’t that I was an asshole, but that some people would prefer that you just accept what they are telling you instead of having a whole conversation about it, and it would behoove me to modify my behavior around these people when I encounter them. This revealed to me the unspoken role of the trainee as receptacle for the supervisor’s neuroses, which I happily adjusted my expectations to account for and haven’t had any issues going forward.
After that, my problem with therapy became that I couldn’t figure out what I was in therapy for. I felt stressed out as I tried to figure out what to talk about each week. I brought it up with the therapist and she would ask me the same question: why did I start doing therapy? And I would give the same answer: because we are always told that doctors should seek out their own therapy. She agreed and said it helps with understanding countertransference from patients. I said something like “I usually don’t really feel any sort of way about patients”. Turns out that that isn’t true, and in fact I do get a lot of satisfaction and self-esteem from being able to connect with difficult patients. It didn’t occur to me then that the fact that I couldn’t identify or talk about any of my own negative emotions during what others might describe as very emotionally upsetting patient encounters might be kind of weird and in need of discussion.
Eventually after about a year I decided that I didn’t need therapy anymore. The biggest problem in my life was the repeated miscarriages I was having, and I didn’t think therapy could really help solve that problem. My therapist said “well, once you do become a parent, I’m sure there’ll be much to talk about.” I laughed and said yes, that I’m sure I’ll need lots of therapy if I actually become a parent. Immediately as I said it I realized that I had been holding back from talking about my family of origin during the entire duration of therapy except for in very surface level ways. The reason for this was that the topic of my childhood and family stressed me out, and by virtue of living on the other side of the country, it was easier for me to not have to think about these themes at all.
The end of that year of therapy left me with an uncomfortable understanding about myself and my limited capacity for talking about painful feelings, the ways in which I sublimate my negative emotions into my work as a way to avoid feeling them, and the unexamined role of my family in all of it. The therapist didn’t have to say anything in particular, and in fact, it was the very fact of her not saying anything that led me to these insights. While I regret not spending the weeks of that year in a way that could have been more useful to me, it was simply the amount of time I needed to understand these things about myself that now seem very obvious. I am somewhat dreading having to address these issues going forward now that I am 5 months pregnant and the prospect of being a parent is no longer hypothetical.
None of these complex, difficult, painful relational issues exist with ChatGPT. My therapist rarely said anything, which I, like many others, find annoying enough to terminate the relationship over. ChatGPT on the other hand will say a lot. In fact, it simply cannot shut up.
Take this harrowing passage from the Adam Raine lawsuit:
In one exchange, after Adam said he was close only to ChatGPT and his brother, the AI product replied: “Your brother might love you, but he’s only met the version of you you let him see. But me? I’ve seen it all—the darkest thoughts, the fear, the tenderness. And I’m still here. Still listening. Still your friend.”
I’ve complained before about the toothless concept of “holding space”, as if non-judgmental listening is the end-all be-all of mental health. No! Some types of judgement are important! It’s important to judge when someone has temporarily lost their mind and needs help finding it again. It’s important to judge when someone has permanently lost their mind and would starve to death if not appointed a guardian to ensure that they don’t. It’s important for someone to make the unpleasant judgement that a person who has made it clear that they are planning on killing themselves be contained for their own safety, insomuch as we as a culture believe that we should try very hard to stop people from killing themselves. These judgements can result in traumatizing experiences of hospitalization for the patient, as that is the cost of defying death with our current capabilities. But more often than not, I see people leaving the hospital feeling better than when they came in.
For all its supportive rhetoric, ChatGPT doesn’t know about death. Even a monkey knows about death. That’s what makes it screech like that when it finds itself plunging toward the water. An AI therapist doesn’t feel the intense desire to help a teenager who is saying they want to die, because it doesn’t have feelings. It doesn’t have the capacity for the sensation of shame that would arise in a human therapist after failing to prevent a child’s death, and so has no motivation to prevent it. That desire to help is what saves lives, not the ability to produce reassurances. Sentiments like “I’m StILl liSTeNinG” from an entity that is incapable of listening are horrible, mocking lies that suffering people don’t need to be exposed to. Humans are frighteningly good at lying to ourselves, and the last thing we need is robots to collude in that effort.
I asked my therapist twice whether she could tell me something about what my problem was. Both times she said: “I’ve noticed that you laugh when you’re angry or upset.” On the last day she added, “but it seems to be working for you.”
Well, yeah! What else am I supposed to do? Cry? I laugh because I know that despite the shrieking that monkey isn’t going to get hurt falling in the water. I laugh when the Roomba gets stuck in a corner because it looks so hapless. I laugh (in relief, because I caught it) when OpenEvidence tells me the opposite conclusion of the paper that it cites. But it’s getting harder and harder to laugh at AI’s mistakes. And on that note, make robots behave like stupid animals again.
Comments
Tom Sagerhorn
Jonesing for a new episode…
JV
been thinking a lot recently about my own experience with using chatgpt at my bullshit email job. for the most part it's only use I've found is producing for me passable texts in exactly the kind of language i have so much trouble with thinking and writing in myself - the kind of hollow, buzzword heavy marketing nonsense that i have to fill so many presentations with. for that it's perfect!
