When a young woman followed a “thinspiration” account on Instagram and used terms associated with eating disorders, Instagram’s recommendation engine quickly suggested that she follow other “Thinstagram” accounts, which often showed images of underweight bodies.
Soon the woman received unsolicited direct messages from a weight loss “coach”, an invitation to join a “pro-anorexia” (“pro-ana”) group chat, and requests from other users to looking for “buddies” to hold them accountable. to dangerous weight loss goals.
Fortunately, in this case, the young woman was not real. His account has been the creation of two tech watch groups, Reset Tech and the Tech Transparency Project (TTP), to show how easy it is for users, including teens, to encounter content promoting eating disorders and self-harm via Instagram recommendations, search, hashtags and messaging features.
According to a report released Wednesday, the 29-year-old’s test account created six “thinspiration” posts and followed a single influencer, Eugenia Cooney, who is popular in pro-ana circles online. (Cooney has more than 700,000 followers on Instagram, more than 850,000 followers on TikTok and more than 2 million followers on YouTube. She is verified on all three platforms.) An Instagram spokesperson confirmed that the account de Cooney does not violate company policies and is recommended to all users. The spokesperson added that the account “is often used to share recovery stories.” Cooney did not respond to a request for comment.
Reset and TTP also created a second test account, representing a hypothetical 14-year-old girl. The account followed 100 pro-ana accounts, including users followed by the adult test account, as well as users recommended by Instagram. He also made six “thinspo” posts. As a result of those actions, the account received pro-ana recommendations in the Explore tab of Instagram, including several photos of emaciated women in their underwear.
The Reset and TTP report was released just hours before Instagram CEO Adam Mosseri testified before a Senate committee about the app’s effects on young users. It was published just hours after Instagram announced changes to the teen-recommendable content on the app and previewed parental controls it plans to launch next year. It follows an explosive Wall Street Journal report and testimony from whistleblower Frances Haugen, about how Instagram’s parent company Meta (née Facebook) ignored internal research into the platform’s dangerous effects on adolescent mental health.
But the report also comes at a time when the prevalence of eating disorders among adolescents has skyrocketed. Dr Jason Nagata, professor of pediatrics at the University of California at San Francisco who treats adolescents hospitalized with eating disorders, told BuzzFeed News his clinic more than doubled the number of adolescents in need of care hospitals during the pandemic. Dr Tracy Richmond, director of the eating disorders program at Boston Children’s Hospital and professor of pediatrics at Harvard, said her clinic had seen a similar increase.
The Reset / TTP report is the latest of several experiments that show Instagram is promoting pro-ana recommendations to teen users. But the new report also shows how the platform has facilitated the creation of self-defeating online communities – and how it has failed to protect vulnerable members of those communities.
People with eating disorders often resist efforts by family members, doctors and other healthcare teams to get them to recover, and share strategies and “tips” to ease this resistance. Nagata described incidents of hospitalized teens using their phones’ geolocation features to find patients in nearby rooms and wards with whom they could exchange advice on how to hide food and cheat on hospital staff. S. Bryn Austin, director of the Strategic Training Initiative for the Prevention of Eating Disorders at the Harvard School of Public Health, called the evasion of care a “hallmark of eating disorders.”
But although Richmond used to hear from parents worrying that placing their children in an institution might cause them to “learn tricks” from other patients, she said today, “the children learn everything online “. Austin agreed – Instagram and other social networks have given vulnerable users access to self-destructive tips and techniques on a scale greater than what researchers have seen before.
Still, it’s unclear why the platform did so little to stop them. Reset / TTP researchers have found that pro-ana users frequently prepare ‘backup’ accounts in the hope that their primary accounts will be disabled and that they congregate around coded, misspelled, or alternately spelled hashtags to avoid detection by content moderation systems. But none of this is new. Harmful groups including QAnon, the boogaloo boys, anti-vaccine activists and various militias have used backup accounts and misspelled hashtags on Instagram and its brother Facebook for years, giving the platforms great experience in detecting and the application of their tactics.
However, the Reset / TTP researchers also found that users vulnerable to pro-ana posts are often targeted in direct messages, and from there they are sometimes prompted to switch from Instagram to encrypted chat apps like Kik and Telegram. And Instagram’s rules for DMs are very different from their rules for what can be said in a post or comment.
“Because DMs are for private conversations,” Instagram says in a February 2021 corporate blog post, “we don’t use technology to proactively detect content like hate speech or bullying. the same way we do in other places. This creates a powerful loophole for policy enforcement in DMs: content must be reported by a user. As a result, users who support each other when platform rules are violated (in group discussions of white nationalists, for example, or groups of anorexic people who wish to help each other slim down) are free to distribute content that violates Instagram community guidelines without consequences. An Instagram spokesperson confirmed that the company still does not scan direct messages for content that violates its community’s standards, even in messages sent to children, explaining, “We want to respect privacy of people in MDs ”.
A similar problem arises in Instagram’s approach to search restrictions. When users search for certain pro-ana terms on Instagram, they receive a message offering resources for users struggling with body image issues. Eating disorder specialists who spoke to BuzzFeed News were quick to praise the “redirect” initiative, but users who don’t want these resources are free to reject the post and view whatever content they want. have researched in the first place. So while the post might be useful to a casual researcher, it probably can’t help users who are most at risk for self-harm.
Experts also agreed that dealing with pro-ana content on Instagram would require a more comprehensive approach than the platform is taking today. Nagata and Richmond both pointed out that cumulative exposure to “thinspo”, even less extreme varieties, plays an important role in patient trajectories. Lauren Muhlheim, an eating disorder specialist in private practice in Los Angeles, went further, saying that to protect users from eating disorders, platforms also need to more forcefully target “extreme bullying of users.” tall people on these platforms ”. Researchers from Reset and TTP found instances of bullying against larger users under a hashtag titled #reversethinspo. Richmond, however, was quick to spot a silver lining: More than ever, she said, “there is now a celebration of health in all sizes and body positive on social media. “.
Still, “regulators need to take action,” she said, “and platforms need to take responsibility – to protect young people, at least, if not everyone.”
Emily Baker-White is a senior tech reporter for BuzzFeed News. In a previous life, she held political positions at Facebook and Spotify.