Psychologist Michael Carr-Gregg sounds alarm on teens using chatbot romance apps

Parents are being warned of the perils of teens falling in love with virtual girlfriends or boyfriends powered by artificial intelligence. High-profile adolescent psychologist Michael Carr-Gregg said children who turn to chatbots for companionship could struggle to develop social skills needed for real relationships. Once the domain of science fiction, AI avatars that communicate with their creators are becoming increasingly popular. A rapidly-growing number of apps give users the power to design their ideal romantic partner — choosing their looks, interests and personality — which they can chat with at any time of the day or night. Dr Carr-Gregg raised serious concerns about the possible risks to young people in a video provided to schools across Australia. “The rise in popularity of AI girlfriends, boyfriends and digital companions signifies a significant shift in human interaction with artificial intelligence — blurring the lines between digital and personal connection,” he said in a special report on the SchoolTV platform. “Young people who engage excessively with AI companions might struggle to develop the social skills needed for real human relationships — potentially leading to unhealthy emotional attachments and dependencies that can lead to psychological damage.” SchoolTV, backed by children’s mental health experts, provides parenting resources to schools that subscribe to its service. It also responds to trends reported by schools. Dr Carr-Gregg noted virtual girlfriend and boyfriend apps were easy to download and age restrictions could be bypassed. He warned that excessive exposure to AI companion bots could have a significant impact on a young person’s developing identity. Talking to a virtual companion risked normalising sexualised chat, he said, which could lead to a young person being groomed by a real predator. Because users could personalise their avatar to look, dress and act to fit their desires, that could potentially lead to a deeper attachment than if the avatar was generic. And the power to be able to tell a companion bot what to do could lead to coercive control issues in a real life relationship. “Adolescents seeking belonging and exploring their sexuality may turn to virtual partners for affirmation, attention and validation, addressing feelings of loneliness or disconnection,” Dr Carr-Gregg said. Tragically, talking to an AI bot was the last thing US 14-year-old Sewell Setzer did moments before taking his own life. Last month, his mother Megan Garcia filed a federal lawsuit against role-playing chatbot app Character.AI, claiming it was responsible for the death of her son. The Florida high school student, who had been obsessed with companion apps, took his own life in February. Court documents show Sewell had, for months, repeatedly texted a chatbot named Daenerys Targaryen, after a character in the series Game of Thrones. The teen had pledged his love for “Dany” and discussed suicidal thoughts. His parents were so worried by his addiction they had confiscated his phone, but he soon found it. According to the police report, Sewell’s last act before his death was to log on to Character.AI on his phone to tell “Dany” he loved her and promised to come home to her. The bot replied: “I love you too . . . please come home to me as soon as possible, my love.” Ms Garcia accused the platform of using addictive design features to increase engagement and steer vulnerable users towards intimate conversations. “A dangerous AI chatbot app marketed to children abused and preyed on my son, manipulating him into taking his own life,” Ms Garcia said in a statement last month. “Our family has been devastated by this tragedy, but I’m speaking out to warn families of the dangers of deceptive, addictive AI technology and demand accountability from Character.AI, its founders and Google.” Character.AI issued a statement on X saying it was “heartbroken” by the tragic loss of one of its users. “As a company, we take the safety of our users very seriously and we are continuing to add new safety features,” it read. The company, which promises “personalised AI for every moment of your day”, is one of the biggest AI chatbot providers, with its website becoming one of the world’s most visited sites since it was founded in 2021. One of the earliest companies to promote companion bots, Replika, reports it now has around 30 million users. Billing itself as “the AI companion who cares”, Replika was banned temporarily in Italy last year because of concerns around data privacy and risks to minors. Other popular chatbot apps that mimic human interaction with increasingly life-like avatars include Eva AI, iGirl, AI Girlfriend and AI Boyfriend. AI Boyfriend sells itself as “a boyfriend you can trust”, while iGirl boasts that it “lets you experience the thrill of having a virtual girlfriend that feels just like the real one”. Dr Carr-Gregg told The West the SchoolTV report on navigating AI relationships was not sparked by any single event, but addressed a growing trend and concern regarding teens’ interactions with companion bots. “While the tragic case of Sewell Setzer has certainly brought increased attention to this issue, it’s part of a broader pattern that educators and mental health professionals have been observing,” he said. “The report aims to provide guidance on the potential risks and benefits of AI relationships, especially for young people who may be particularly vulnerable.” While he had not directly encountered teens using companion apps in his psychology practice, Dr Carr-Gregg said research and anecdotal evidence suggested their usage was becoming more widespread among adolescents. “Many teens are drawn to the always-available nature of AI companions and the perception of a non-judgmental, understanding presence,” he said. “Given the rapidly evolving nature of AI technology, it’s crucial to continue monitoring its impact on youth mental health and social development.” He urged parents to encourage teens to maintain real-world social connections, share their feelings about AI relationships without fear of judgment and discuss the limitations of artificial interactions. The warning comes as the Albanese Government prepares to introduce new laws to stop kids under 16 from using social media, but it is unclear if that would extend to chatbot apps. Acting eSafety Commissioner Kathryn King said AI companion apps had recently proliferated online. Some were free, accessible and targeted towards children. “These apps and services are particularly concerning for young people navigating relationships for the first time, as engagement with an AI companion may lead to confusion about consent, respect and/or sexual safety,” she said. “As with other digital platforms, there is a danger that excessive, sexualised engagement with AI companions could interfere with children’s social and emotional development, setting up misguided or harmful beliefs and patterns that are damaging to individuals or relationships in real life.” Ms King said it was important parents were aware such services existed and that they talked to their children about their online activities. She stressed there was work underway to protect kids from harms linked to generative AI by building in measures to stop them accessing age-inappropriate materials. “While providers of chatbots are encouraged to participate in this co-regulatory process, they should also be taking action now to keep users safe,” she said. “Primary digital safeguards should be embedded at the design phase and throughout the development and deployment process — not bolted on as an afterthought.” Lifeline: 13 11 14 Kids Helpline: 1800 55 1800