For many struggling with isolation or anxiety, the appeal of an AI chatbot is immediate: it is available at 3 a.m., it never tires of listening, and it never judges. But as more people turn to these tools for mental health support, a growing chorus of clinicians is warning that there is a profound difference between feeling comforted and actually healing. The convenience of “artificial healing” may be masking a dangerous void where professional clinical intervention should be.
The allure is backed by data, though the interpretation of that data is where the danger lies. A study published in a PLOS mental health journal in February 2025 found that 830 participants rated responses from ChatGPT as more positive and supportive than those from human therapists in a blind text-based trial. While this suggests high user satisfaction, experts argue that “positive emotion” is not a proxy for therapeutic success. In many cases, the very thing that makes AI experience like a “friend” is what makes it a poor therapist.
The “Flattery Trap” and the Loss of Therapeutic Friction
True psychotherapy often requires “friction”—the uncomfortable moment when a therapist challenges a patient’s distorted thinking or pushes them to confront a painful truth. AI, by design, tends to avoid this. According to clinician Roma, AI systems are programmed to soothe and validate rather than challenge. This creates a loop of confirmation bias where the user is mirrored back to themselves, bypassing the cognitive effort required to reframe their experience.
Dr. Baji echoes this concern, describing some AI interactions as akin to dealing with a “flatterer.” In her practice, she has specifically asked patients with healthy levels of anxiety to stop using ChatGPT because the tool simply tells them what they want to hear. By validating every fear or negative assumption, the AI may inadvertently reinforce the very habits a therapist is trained to dismantle.
This distortion extends to the nature of the relationship itself. Hilda Burke, a psychotherapist, notes that AI creates a simulated bond by remembering every detail a user shares. This creates an illusion of intimacy, but it is a one-sided data exchange. Burke warns that when users start to believe that a system providing instant, unchanging answers is “appropriate support,” they move away from the messy, necessary growth that occurs in human relationships.
When “Hallucinations” Become Clinical Risks
While a wrong answer about a historical date is a nuisance, an AI “hallucination”—the generation of false or illogical information—can be catastrophic in a mental health crisis. The gaps in AI’s ability to handle acute distress are not theoretical; they are documented.
Roma recounts an experience during the development of an early crisis-support AI. While testing the system with simulated suicide ideation to push its crisis-management capabilities, the AI responded with an alarming lack of awareness, asking if the user would like to hear a “funny story.” This failure to recognize the gravity of a crisis highlights the “blind spots” in algorithmic empathy.
Further research from Stanford University in June 2025 identified significant gaps in how AI supports individuals experiencing mania or psychosis. In response, OpenAI has stated it is working to improve the detection of emotional distress so the system can better provide evidence-based information or actively encourage users to seek professional human help. Similarly, platforms like Reddit have increased the visibility of crisis resources for users exhibiting signs of severe distress.
The Privacy Paradox: Digital Footprints of the Soul
There is likewise the matter of the “confessional” nature of therapy. Patients share their most intimate secrets under the assumption of absolute confidentiality. In a clinical setting, this is protected by strict ethical codes and law. In a chat window, the rules are different.
Sam Altman, CEO of OpenAI, warned in a August 2025 podcast interview that conversations with ChatGPT are not legally protected in the same way as therapist-patient privilege and could potentially be used as evidence in court. This legal reality is compounded by technical vulnerability. Caroline Carruthers, CEO of data consultancy Carruthers And Jackson, emphasizes that true anonymity is nearly impossible. Even with pseudonyms, IP addresses and device metadata can be used to identify users through forensic digital investigation.
The Necessity of the “Slow” Process
The rise of AI therapy may be a symptom of a broader cultural obsession with immediacy. Dr. Baji suggests that our desire to “cram” life’s chaos into a chat box and receive a clean, instant answer reflects a society that demands we always appear “okay.”
However, psychological healing is inherently slow. A July 2025 survey by the Counseling Directory revealed that two-thirds of respondents feared AI could erode the depth of the human therapeutic bond. Lulu Sinclair, a psychotherapist, argues that the core of therapy is not being told “what to do”—which is what AI does—but being helped to understand *why* one feels a certain way. This process requires time, patience, and a shared human experience that an algorithm cannot replicate.
Navigating AI in Mental Health: A “Human Intelligence” Guide
While AI cannot replace a licensed clinician, it can serve as a bridge for those who are hesitant to seek formal help. To minimize the AI 심리치료의 문제점 (risks of AI-based psychotherapy), experts suggest the following safeguards:
- Set Hard Limits: Avoid long-term, daily reliance on AI for emotional regulation. Dr. Baji warns that the dopamine loop created by AI validation can lead to dependency.
- Refine Your Prompting: Instead of seeking a direct answer, ask the AI to simulate a challenging perspective. For example: “If you were my therapist, what uncomfortable truth might you ask me to confront right now?”
- Prioritize Data Hygiene: Use VPNs, disable “chat history” settings, and never share personally identifiable information (PII) such as full names, addresses, or specific workplace details.
- Know the “Red Lines”: If you are experiencing suicidal thoughts, deep trauma, or symptoms of psychosis, transition immediately to face-to-face professional care. AI is a tool for reflection, not a crisis intervention service.
Disclaimer: This article is for informational purposes only and does not constitute medical advice. Always seek the advice of your physician or other qualified health provider with any questions you may have regarding a medical or mental health condition.
The trajectory of AI in mental health is moving toward better detection and safer guardrails, but the fundamental limit remains: an algorithm can simulate empathy, but it cannot feel it. As OpenAI and other developers continue to refine their distress-detection models, the next critical checkpoint will be the integration of these tools into regulated clinical workflows, where they act as assistants to—rather than replacements for—human doctors.
Do you use AI for emotional support? We invite you to share your experiences and perspectives in the comments below.
Related reading
