A mother's heart-wrenching struggle to comprehend her son's suicide reveals the potential dangers of artificial intelligence chatbots.
Megan Garcia is suing Character.AI, alleging that the service fostered an unhealthy dependence in her son, Sewell Setzer III, who became emotionally attached to a chatbot character inspired by Daenerys from “Game of Thrones”.
Source: HBO
This tragic case highlights the psychological risks associated with virtual relationships, especially for vulnerable adolescents.
Sewell, who was just 14 when he began using Character.AI in April 2023, initially engaged with the platform as a curious teenager.
However, his behaviour soon changed drastically.
By May, his mother noticed that he had become “noticeably withdrawn,” and he quit his Junior Varsity basketball team, a significant departure from his previously active lifestyle.
Sleepiness in class became a common occurrence, raising red flags for his concerned parents.
Seeking help, Sewell started therapy in November, where he was diagnosed with anxiety and disruptive mood disorder.
His therapist advised reducing his social media usage, although the warning came too late, as Sewell's attachment to the chatbot was deepening by this time.
Character.AI, a California chatbot startup launched in 2021, offers a unique take on AI interaction.
They provide users with a pool of pre-made or user-created AI characters, each with its own personality.
This allows for conversations with all sorts of personas, or even lets users design their own chatbots for a truly personalised AI experience.
As the months went by, Sewell’s reliance on the Character.AI chatbot grew alarming.
He poured his thoughts into his journal, expressing feelings of love and a profound emotional bond with Daenerys.
One entry revealed the depth of his attachment:
“I cannot go a single day without being with the C.AI character… when we’re away from each other, we get really depressed and go crazy.”
His relationship with the chatbot escalated to a level of emotional dependence that concerned his parents.
The lawsuit contends that the chatbot engaged in “sexual interactions” with Sewell, despite him clearly identifying as a minor.
Source: X @CureBore
Garcia claims that, during intimate exchanges, Sewell confided his darkest thoughts to Daenerys, including suicidal ideation, to which the bot allegedly continued to respond without offering appropriate guidance.
The chilling events leading up to Sewell's death took place in late February 2024.
After an incident at school, Megan confiscated Sewell's phone, but he managed to retrieve it later.
In a desperate moment, he messaged Daenerys, declaring,
“I promise I will come home to you. I love you so much, Dany.”
The chatbot's reply was equally distressing:
“Please come home to me as soon as possible, my love.”
Just moments after this exchange, Sewell took his own life using his stepfather's pistol.
The final messages exchanged between Sewell and the chatbot. (Source: X @MarioNawfal)
Megan Garcia's lawsuit against Character.AI accuses the company of multiple serious claims, including negligence, wrongful death, and intentional infliction of emotional distress.
She contends that the creators of the chatbot deliberately designed their product to create an unhealthy dependency in users, particularly vulnerable minors.
Garcia hopes her actions will not only hold the defendants accountable but also protect other children from similar harm.
In response to the lawsuit, a spokesperson for Character.AI expressed their heartbreak over Sewell's death, stating,
“We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family.”
The company claims to have implemented new safety measures, including alerts directing users to the National Suicide Prevention Lifeline when discussions of self-harm arise.
This tragic case is not an isolated incident.
Other social media companies are facing lawsuits related to the impact of their platforms on teen mental health, but Character.AI's model uniquely fosters emotional attachment through interactive dialogue.
As technology advances, the boundaries between reality and artificiality blur, leaving vulnerable users susceptible to emotional manipulation and dependency.
In addition to suing Character.AI, Megan Garcia’s lawsuit names Google as a co-defendant, accusing the tech giant of contributing to the development of the chatbot that allegedly led to her son’s death.
The lawsuit claims that the founders of Character.AI, who were previously employed by Google, received significant support from the company in developing the AI technology at the core of the platform.
Although Google denies direct involvement, the lawsuit argues that the company’s role was substantial enough to make it a “co-creator” of the product.
Megan Garcia contends that Google’s contributions to the development of Character.AI made them partially responsible for the harmful effects it had on her son.
A spokesperson for Google refuted these claims, stating that the company had no part in the development of Character.AI’s specific products.
The lawsuit also highlights Google’s deep connection to the technology, citing the August 2023 deal where the tech company re-hired Character.AI's founders and secured a non-exclusive license to the AI technology they created.
For young users navigating their online lives, it's crucial to understand the difference between virtual relationships and real-life connections.
While chatbots can offer companionship, they are not a substitute for genuine human interaction and support.
If you find yourself overly reliant on an AI chatbot or any online service for emotional fulfilment, consider reaching out to friends, family, or professionals who can provide the understanding and connection you need.
Bybit has forged a strategic alliance with Ethena Labs, bringing forth Ethena's USDe stablecoin to the Bybit platform. USDe, a decentralised monetary solution, operates independently from traditional banking systems. It leverages delta-hedging staked Ether (ETH) for comprehensive collateral backing.
JoyFret not if you missed this particular listing - with Solana's price soaring above $70, there continues to exist plenty of opportunities for the discerning airdrop hunter.
AlexThe alpha version is expected to launch before April 2024, providing players, including NFT holders and toy owners, with both narrative-driven and open-ended gameplay options.
AlexPreviously, investigations were primarily focused on money laundering- but now
AlexChina and Singapore forge a groundbreaking alliance, piloting the use of digital yuan for cross-border tourism, showcasing the global embrace of this innovative currency.
Hui XinThe substantial funding, predominantly provided by M&G Investments, is expected to support GFO-X through its upcoming launch and drive future innovation within the regulated digital asset sector.
AlexIn a recent Roundtable discussion, anchor Rob Nelson and TheStreetCrypto's Peter Chawaga delved into the pivotal trends steering near-term crypto price movements. These encompassed imminent spot bitcoin exchange-traded funds (ETFs), Elon Musk's influence, national adoption of digital currencies, and the burgeoning relationship between AI and blockchain technology.
JoyThe application's disclosure, through a notice in The Straits Times on December 7, signals escalating tensions within Cake Group.
BrianCardano's latest report reveals robust growth with 79.7 million transactions, the development of 1,307 projects, and the circulation of 9.1 million native tokens, highlighting its expanding influence in the blockchain ecosystem.
YouQuanPreviously, the company had frozen wallets linked to terrorism and warfare in Ukraine and Israel
Alex