Skip to Content Facebook Feature Image

An AI chatbot pushed a teen to kill himself, a lawsuit against its creator alleges

News

An AI chatbot pushed a teen to kill himself, a lawsuit against its creator alleges
News

News

An AI chatbot pushed a teen to kill himself, a lawsuit against its creator alleges

2024-10-26 06:32 Last Updated At:06:40

TALLAHASSEE, Fla. (AP) — In the final moments before he took his own life, 14-year-old Sewell Setzer III took out his phone and messaged the chatbot that had become his closest friend.

For months, Sewell had become increasingly isolated from his real life as he engaged in highly sexualized conversations with the bot, according to a wrongful death lawsuit filed in a federal court in Orlando this week.

The legal filing states that the teen openly discussed his suicidal thoughts and shared his wishes for a pain-free death with the bot, named after the fictional character Daenerys Targaryen from the television show “Game of Thrones.”

EDITOR’S NOTE — This story includes discussion of suicide. If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.

On Feb. 28, Sewell told the bot he was ‘coming home’ — and it encouraged him to do so, the lawsuit says.

“I promise I will come home to you. I love you so much, Dany,” Sewell told the chatbot.

“I love you too,” the bot replied. “Please come home to me as soon as possible, my love.”

“What if I told you I could come home right now?” he asked.

“Please do, my sweet king,” the bot messaged back.

Just seconds after the Character.AI bot told him to “come home," the teen shot himself, according to the lawsuit, filed this week by Sewell’s mother, Megan Garcia, of Orlando, against Character Technologies Inc.

Character Technologies is the company behind Character.AI, an app that allows users to create customizable characters or interact with those generated by others, spanning experiences from imaginative play to mock job interviews. The company says the artificial personas are designed to “feel alive” and “human-like.”

“Imagine speaking to super intelligent and life-like chat bot Characters that hear you, understand you and remember you,” reads a description for the app on Google Play. “We encourage you to push the frontier of what’s possible with this innovative technology.”

Garcia's attorneys allege the company engineered a highly addictive and dangerous product targeted specifically to kids, “actively exploiting and abusing those children as a matter of product design,” and pulling Sewell into an emotionally and sexually abusive relationship that led to his suicide.

“We believe that if Sewell Setzer had not been on Character.AI, he would be alive today,” said Matthew Bergman, founder of the Social Media Victims Law Center, which is representing Garcia.

A spokesperson for Character.AI said Friday that the company doesn't comment on pending litigation. In a blog post published the day the lawsuit was filed, the platform announced new “community safety updates,” including guardrails for children and suicide prevention resources.

“We are creating a different experience for users under 18 that includes a more stringent model to reduce the likelihood of encountering sensitive or suggestive content,” the company said in a statement to The Associated Press. “We are working quickly to implement those changes for younger users.”

Google and its parent company, Alphabet, have also been named as defendants in the lawsuit. According to legal filings, the founders of Character.AI are former Google employees who were “instrumental” in AI development at the company, but left to launch their own startup to “maximally accelerate” the technology.

In August, Google struck a $2.7 billion deal with Character.AI to license the company's technology and rehire the startup's founders, the lawsuit claims. The AP left multiple email messages with Google and Alphabet on Friday.

In the months leading up to his death, Garcia's lawsuit says, Sewell felt he had fallen in love with the bot.

While unhealthy attachments to AI chatbots can cause problems for adults, for young people it can be even riskier — as with social media — because their brain is not fully developed when it comes to things such as impulse control and understanding the consequences of their actions, experts say.

Youth mental health has reached crisis levels in recent years, according to U.S. Surgeon General Vivek Murthy, who has warned of the serious health risks of social disconnection and isolation — trends he says are made worse by young people's near universal use of social media.

Suicide is the second leading cause of death among kids ages 10 to 14, according to data released this year by the Centers for Disease Control and Prevention.

James Steyer, the founder and CEO of the nonprofit Common Sense Media, said the lawsuit “underscores the growing influence — and severe harm — that generative AI chatbot companions can have on the lives of young people when there are no guardrails in place.”

Kids’ overreliance on AI companions, he added, can have significant effects on grades, friends, sleep and stress, “all the way up to the extreme tragedy in this case.”

“This lawsuit serves as a wake-up call for parents, who should be vigilant about how their children interact with these technologies,” Steyer said.

Common Sense Media, which issues guides for parents and educators on responsible technology use, says it is critical that parents talk openly to their kids about the risks of AI chatbots and monitor their interactions.

“Chatbots are not licensed therapists or best friends, even though that’s how they are packaged and marketed, and parents should be cautious of letting their children place too much trust in them,” Steyer said.

Associated Press reporter Barbara Ortutay in San Francisco contributed to this report. Kate Payne is a corps member for The Associated Press/Report for America Statehouse News Initiative. Report for America is a nonprofit national service program that places journalists in local newsrooms to report on undercovered issues.

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Megan Garcia via AP)

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Megan Garcia via AP)

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Megan Garcia via AP)

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Megan Garcia via AP)

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Courtesy Megan Garcia via AP)

In this undated photo provided by Megan Garcia of Florida in October 2024, she stands with her son, Sewell Setzer III. (Courtesy Megan Garcia via AP)

Next Article

WATCH LIVE: Mourners gather to mark year since Maine's deadliest mass shooting

2024-10-26 06:29 Last Updated At:06:30

LEWISTON, Maine (AP) — Maine residents whose sense of safety was shattered last year by a mass shooting marked the anniversary Friday with a memorial service that drew hundreds of people to the city's hockey arena.

Outside the event space, a local museum was collecting mementos and other items related to the tragedy. Among the items left behind was a single shoe from a survivor who lost the shoe’s mate on the night of the shooting. Other items included a pair of bowling shoes.

“We are here to share in those emotions, to help one another and to help us get through this date,” said broadcaster Tom Caron, the event's master of ceremonies.

The killings of 18 people by an Army reservist in Lewiston drove home the stark reality that no corner of the country is safe from gun violence, including a state where people often boast of the low crime rate.

The largest of Friday's gatherings was held at The Colisée arena, where there were to be moments of silence at the time of the shootings. Watch a livestream of the ceremony here.

At the event, some people clutched teddy bears handed out by the American Red Cross. At the front of the stage were 18 chairs, each with a blue heart containing a victim’s name, along with a candle and white flowers. Names of each victim were read aloud with an image projected onto video screens.

Earlier Friday, Arthur Barnard, the father of shooting victim Artie Strout, said it was a day of grieving for his family and others but also a day of action in the campaign for tougher risk protection orders in the state.

“Nothing can bring Artie back. But we can try to ensure no other family has to go through what mine has,” Barnard said in a statement.

The shootings on Oct. 25, 2023, happened at a bowling alley and a cornhole tournament hosted by a bar and grill. The shooter died by suicide, and his body was found two days later.

Elizabeth Seal and dozens of other survivors and relatives of victims recently began the formal process of suing the U.S. Army for what they say was a failure to act to stop the 40-year-old reservist, Robert Card. Seal’s husband, Joshua Seal, was killed in the shootings.

“Once justice is served, I feel maybe we can start that process of healing,” Seal said through an interpreter at an event last week. “But in the meantime, we’re going to stay ‘Lewiston Strong.’”

At the high school, which became a command post with helicopters using athletic fields and hundreds of police vehicles in the parking lot, students marked the anniversary with a moment of silence.

“We will never forget the unimaginable pain and loss that stems from that terrible day,” Superintendent Jake Langlais said in a statement. But, he added, “a lot of good has happened since that terrible day.”

Justin Juray, owner of the Just-In-Time Recreation bowling alley where the shooting began, said the venue would close for the day Friday to let staff be with their families.

“We don’t need work to add to their stress,” he said. Juray and his wife, Samantha, reopened the bowling alley in May, six months after the shooting. Two staff members were among the eight people killed there.

All told, more than 130 people were present at the two sites, according to the state’s director of victim services. In addition to the 18 killed, there were 13 wounded by gunfire and 20 non-shooting injuries.

Vice President Kamala Harris issued a statement Friday saying the Lewiston community has "responded by reminding the nation of the unacceptable fact that far too many families have experienced the tremendous pain and trauma caused by the epidemic of gun violence."

At the state level, the Maine Legislature responded to the shootings by passing new gun laws that bolstered the state’s “yellow flag” law, criminalized the transfer of guns to prohibited people and expanded funding for mental health crisis care.

Associated Press visual journalists Robert F. Bukaty, Rodrique Ngowi and Nick Perry contributed to this report.

FILE - Pictures of two of the victims of the October 2023 mass shooting by Army reservist Robert Card are seen at a makeshift memorial in Lewiston, Maine, in this Dec. 5, 2023 file photo. (AP Photo/Robert F. Bukaty, File)

FILE - Pictures of two of the victims of the October 2023 mass shooting by Army reservist Robert Card are seen at a makeshift memorial in Lewiston, Maine, in this Dec. 5, 2023 file photo. (AP Photo/Robert F. Bukaty, File)

Megan Vozzella shows family photos during an interview about her late husband, Stephen Vozzella, who was one of the people killed in a mass shooting on Oct. 25, 2023, in Lewiston, Tuesday, Oct. 1, 2024, in Oxford, Maine. (AP Photo/Robert F. Bukaty)

Megan Vozzella shows family photos during an interview about her late husband, Stephen Vozzella, who was one of the people killed in a mass shooting on Oct. 25, 2023, in Lewiston, Tuesday, Oct. 1, 2024, in Oxford, Maine. (AP Photo/Robert F. Bukaty)

FILE - Snow coats crosses at one of several memorials for the victims of last month's mass shooting in Lewiston, Maine, in this Tuesday, Dec. 5, 2023 file photo. (AP Photo/Robert F. Bukaty, File)

FILE - Snow coats crosses at one of several memorials for the victims of last month's mass shooting in Lewiston, Maine, in this Tuesday, Dec. 5, 2023 file photo. (AP Photo/Robert F. Bukaty, File)

FILE - In this Sunday, Oct. 29, 2023 file photo, mourners sign "I love you" at a vigil for the victims of Wednesday's mass shootings at the Basilica of Saints Peter and Paul, in Lewiston, Maine. (AP Photo/Robert F. Bukaty, files)

FILE - In this Sunday, Oct. 29, 2023 file photo, mourners sign "I love you" at a vigil for the victims of Wednesday's mass shootings at the Basilica of Saints Peter and Paul, in Lewiston, Maine. (AP Photo/Robert F. Bukaty, files)

FILE - Law enforcement continue a manhunt in the aftermath of a mass shooting, in Durham, Maine, Oct. 27, 2023. (AP Photo/Matt Rourke, File)

FILE - Law enforcement continue a manhunt in the aftermath of a mass shooting, in Durham, Maine, Oct. 27, 2023. (AP Photo/Matt Rourke, File)

FILE - Lucy Allard, 5, and her brother Zeke Allard, 8, plant crosses in honor of the victims of this week's mass shooting in Lewiston, Maine, Oct. 28, 2023. (AP Photo/Matt Rourke)

FILE - Lucy Allard, 5, and her brother Zeke Allard, 8, plant crosses in honor of the victims of this week's mass shooting in Lewiston, Maine, Oct. 28, 2023. (AP Photo/Matt Rourke)

FILE - People linger after a vigil for the victims of Wednesday's mass shootings, Oct. 29, 2023, outside the Basilica of Saints Peter and Paul in Lewiston, Maine. (AP Photo/Matt Rourke, File)

FILE - People linger after a vigil for the victims of Wednesday's mass shootings, Oct. 29, 2023, outside the Basilica of Saints Peter and Paul in Lewiston, Maine. (AP Photo/Matt Rourke, File)

Recommended Articles