Opinion How dangerous is a chatbot as a ‘digital friend’

Originally published at Dutch newspaper Het Parool: https://www.parool.nl/columns-opinie/opinie-hoe-gevaarlijk-is-een-chatbot-als-digitaal-vriendje~b7d55bd4/

The idea that artificial intelligence could someday destroy our world is frightening, but it distracts from a much more pressing issue, argue Thijs Pepping, Menno van Doorn, and Sander Duivestein: the increasing popularity of chatbots like MyAI from Snapchat. ‘The intentions behind MyAI are not wholesome.’

Thijs Pepping and Menno van Doorn and Sander Duivestein, June 3, 2023, 03:00

Digital 'friends' such as Snapchat's MyAI are not harmless toys, argue Thijs Pepping, Menno van Doorn and Sander Duivestein. IMAGE SNAPCHAT

Last Tuesday, another warning against the dangers of artificial intelligence was issued. With one sentence, hundreds of experts, including Sam Altman, the CEO of OpenAI, the company behind ChatGPT, and ‘AI godfather’ Geoffrey Hinton, hope to wake up the world: ‘Reducing the risk of destruction by AI should be a global priority alongside other risks such as pandemics and nuclear war.’

The danger of these kind of dystopian predictions is that they distract from the actual reality.

Artificial intelligence is already everywhere, and if we want to keep a grip on the future, we need to start actually seeing what’s happening right under our noses. The technology of the same Sam Altman, for example, is now being used to harvest data from 13-year-olds via Snapchat’s MyAI. The goal of the friendly chatbot? Generate income through hyper-personalized ads.

This is not an innocent toy
In April, Snapchat made its GPT-powered chatbot accessible for free. The company is banking on young people engaging with artificial intelligence. With the name of this new service, MyAI, they emphasize the personal nature of these conversations.

Much has been said and written about the shaping and addictive character of social media on young people. With AI in the foreground, it gets a notch worse. Where social media went for the eyeballs, the AI friends go for occupying a brain position and competing with human contacts. It’s high time to really pull out all the stops, because this is not innocent teenage play.

MyAI appears as a chat contact among the normal people in the app. Initially, the face of MyAI is neon green or purple with a contrasting neon hairstyle. It screams to be customized, and luckily it can be, allowing the user to choose between an outfit from, for example, Nike, The North Face, or Levi’s. It’s becoming clearer that anyone who wants to raise their children sustainably has to fight against the marketing machines of big tech companies.

Endless scrolling MyAI, whose name you can change as you please, asks how you are doing and actively asks for your opinion on certain things. This positions the bot as a friend rather than a search aid or productivity tool.

While social media timelines show endless new posts, the infamous infinite scroll, this friend will always be there for you, be more patient than friends, and get to know you better with every interaction. And it never forgets anything. The fact that Snapchat’s MyAI will be the first artificial friend for many young people is of course high-level chess. If young people become addicted to their AI and grow up with it now, will other companies still have a chance at a later age?

Snapchat has had a tough time recently due to competitors like TikTok, Instagram, and BeReal. After the company saw its stock price plummet last year and had to lay off 20 percent of its employees, this seems to be a masterstroke. Customizing MyAI with clothes from well-known brands is a smart marketing trick.

Harvesting data

But there is more for Snapchat to gain and the company is open about it: ‘Your data is used to improve Snap’s products and personalize your experience, including ads,’ they state on their MyAI page. This is a new level of data harvesting: developing a product that actively asks for the opinions and feelings of young people, and then using this information for serving hyper-personalized ads.

So the intentions behind MyAI are not clean. The hyper-personal way Snapchat gets to know the children is used for increasingly refined marketing purposes. The effects of this on the adolescent brain take a back seat. MyAI uses ChatGPT. Various companies, from J.P. Morgan to Samsung, currently have a ban on this due to privacy. And last month, it was revealed that adults were engaging in love relationships with the chatbot Replika. When there was an update, they were in turmoil. Some felt as though their partner had suffered ‘traumatic brain injury’ and was no longer the same.

Furthermore, MyAI had already been criticized when it was originally released in March as a companion. In tests by the Washington Post, the bot cheerfully advised a reporter claiming to be 13 years old on how to plan a ‘surprise trip’ with her ’30-year-old boyfriend’, where they were planning to have sex for the first time. According to Snapchat, the bot will now be more cautious in giving age-appropriate advice after the testing period.

Guinea Pigs

The warning against destruction by AI is sensational, but should not distract from the daily AI issues. We are currently the guinea pigs in a worldwide experiment by the big tech companies, and we do not know, for example, what the consequences are of a long-term intimate relationship with an AI. What extra does this technology bring, but above all: what does it take away?

So let’s not get too distracted by futuristic scenarios, but take the daily reality, such as this step by Snapchat, seriously by being on top of it this time. Don’t let the young people swim alone in the deep end. Start a conversation. With your own MyAI to experiment, but of course also with your child to learn together. This way, we don’t have to wait in fear, but we can actually get a grip on our future with AI today.

Thijs Pepping, Menno van Doorn, and Sander Duivestein are trend analysts at VINT – SogetiLabs’ Research Institute of ICT company Sogeti.