In an alarming development, South Korean women and girls are increasingly becoming victims of deepfakes, a rapidly growing form of digital sex abuse. Recent investigations by local media and crowd-sourced efforts have exposed numerous chat rooms on the messaging app Telegram, where artificial intelligence-generated fake sexual images and videos are being distributed. The South Korean authorities have taken notice and are now scrambling to respond to this disturbing trend.

The Korean National Police have initiated an investigation into Telegram over potential charges of aiding and abetting the spread of sexually explicit deepfakes on its platform. This marks the first time that South Korean law enforcement has scrutinized the company, following founder Pavel Durov's arrest and indictment in France last month for alleged illegal activities on the platform. While a Telegram spokesperson has confirmed the company's active efforts in removing reported content that violates its terms of service, concerns about its role in facilitating the distribution of deepfakes persist.

Deepfake victims in South Korea are overwhelmingly women and teenage girls, according to journalists and activists who have been monitoring various chat rooms. Perpetrators source unsuspecting victims' images from social media without their knowledge or consent, while others secretly capture photos of women in private settings. These images are then manipulated using artificial intelligence and shared with strangers or individuals acquainted with the victim through Telegram. Some chat rooms, labeled as "mutual acquaintance rooms" or "humiliation rooms," have reportedly amassed thousands of participants.

The scale of these chat rooms and the extent of the abuses they perpetuate remain unclear. Many of these rooms operate as closed communities, accessible only through invitation links or with permission from the administrator. However, since media attention and activism efforts have exposed their existence, some of these rooms have shut down.

Responding to concerns, Telegram's CEO, Pavel Durov, acknowledged the platform's growing vulnerability to abuse and expressed the company's commitment to finding the right balance between privacy and security. Nevertheless, there has been a sharp increase in digital sex crimes involving deepfakes in South Korea. The government's media watchdog reported nearly 6,500 requests to address sexually abusive deepfake videos between January and July this year – four times the volume compared to the same period in 2020. Additionally, police data reveals an alarming rise, with 297 cases reported in the first seven months of 2021, compared to 180 cases in all of 2020.

Tragically, many of the victims and perpetrators are teenagers. Among the 178 suspects apprehended during the seven-month period, 74% were between the ages of 10 and 19, representing an increase from 65% in 2020. Shockingly, more than half of the deepfakes identified and removed by the government-run Advocacy Center for Online Sexual Abuse Victims involved minors.

South Korea has long struggled with sex crimes, including illegal filming, nonconsensual dissemination of explicit images, online grooming, and sexual blackmailing. Experts in online sex crimes point out that the creators of deepfakes on Telegram often target women they personally know, rather than random strangers. The damage inflicted by these assaults goes beyond privacy violations, resulting in a loss of trust and fractured social relationships for the victims.

Chang Dahye, a research fellow at the Korean Institute of Criminology and Justice, explains that the purpose behind sexually abusive deepfakes often involves belittling women and expressing misogynistic views. These deepfakes typically highlight identifiable faces alongside verbal sexual harassment, serving as platforms for men to affirm and perpetuate their derogatory attitudes towards women.

Women's rights groups in South Korea released a joint statement placing the blame on deepfake crimes firmly on the shoulders of the Yoon Suk Yeol government, accusing it of failing to recognize the underlying sexism fueling these abuses. However, the government has taken steps to address the issue, with a multiagency emergency task force and the governing People Power Party pledging to strengthen investigations, increase punishment for deepfake crimes, and provide enhanced support for the victims.

Nevertheless, the current legal framework for addressing digital sex abuse in South Korea struggles to keep pace with rapidly evolving technologies. Perpetrators often evade punishment, with an arrest rate of only 48% for offenses related to fake sexual materials. Even when these cases go to court, approximately half of the perpetrators receive suspended sentences. Chang Dahye highlights the ongoing challenge of convincing the legal system to recognize the severity of digital sex abuse as it is not perceived to be on par with physical sexual violence.

With the prevalence of deepfakes on the rise and their damaging impact on victims, it is evident that comprehensive efforts to combat this form of digital sex abuse are urgently needed. Striking a balance between privacy and security, strengthening legal frameworks, and prioritizing support for victims will be crucial in the ongoing fight against this insidious threat.