Regulation of generative AI-driven email attacks and pornographic images and videos lags behind

In 2023, generative AI tools assisting business email compromise (BEC), as well as the so-called “WormGPT” for creating malware, were released, and they have already been offered on the dark web. For BEC, scam emails written in Japanese by foreigners often used to sound unnatural and give a sense of awkwardness; however, they can now be composed in natural Japanese with the help of generative AI. There is a risk that deceptive scam emails, indistinguishable from genuine ones based on the text alone, will increase in the future.

In the past, creating malware such as viruses and worms required certain knowledge and skill. With generative AI, however, even those without expertise can easily create it. When trained on past and current malware, generative AI will be able to create programs easier to exploit and likely to become more sophisticated. If WormGPT becomes more widespread, the range of attacks will expand, its features specialized for exploitation will become smarter, and impersonation will be easier.

The Japanese government is currently considering regulation; however, if public agencies are also prohibited from using generative AI to create offensive programs capable of countering WormGPT, it will be difficult to implement countermeasures. While legislation will progress in the future, the question of where to draw the line on regulation is likely to remain.

The legal interpretation of generative AI-driven pornographic images and videos also presents an issue. Pornography has been in high demand and has long contributed significantly to the technological development of computers, the Internet, and information and communication technology (ICT). In the past, there were cases where uncensored images and videos were released on websites, which led to arrests. That said, methods have become more sophisticated – for example, some content is released with restorable pixelation. Today, AI makes it easier than ever to restore pixelated images and videos to their original state.

One of the problems with generative AI is that it can easily fabricate pornographic images. While analog methods, such as cutting out a face from a photo and pasting it onto another image, are easy to detect as fabrications, generative AI can create smooth images of people, making it susceptible to misuse. If someone uses generative AI to create and publish a nude photo of a real person, they are likely to be charged with defamation.

Deepfakes, which cause social disruption, can only be prosecuted under defamation and obstruction of business

Generative AI-driven deepfakes are also a problem. While creating them itself is unlikely to be a problem, spreading them is subject to criminal law.

In the case of deepfakes, there is a considerable risk of prosecution not only for defamation but also for obstruction of business. This is because spreading deepfake images and videos, including during disasters, may hinder the operations of companies and municipalities. Similar to falsely reporting a bomb threat, broadcasting images or videos that appear to show a fire, for example, can be punishable as obstruction of business since people may be forced to respond for a certain period of time.

On top of that, there is also a potential for incitement. Rather than provoking with words, using images and videos can increase the sense of reality, which may lead to unrest. Deepfakes have also become a problem in recent elections, and election interference could also occur. However, the discussion is still ongoing, and ultimately, anything is allowed now as long as they do not fall under obstruction of business or defamation.

Now, you might think it would be better to create a law as a precaution to prevent problems from arising in the first place. However, in reality, this is quite difficult. This is because criminal law can only be enacted if there is a social need for regulation.

We still have no choice but to be careful when regulating expressive acts. If regulation goes too far, freedom of expression will be violated; therefore, other countries have not yet imposed strict regulations either. In Europe, where many people emphasize social solidarity, regulations are likely to become stricter, while in Japan and the United States, the focus seems to be on protecting freedom of expression.

Legal interpretation of realistic pornography created by generative AI, especially child pornography, is challenging

One of the more troubling issues with generative AI concerns child pornography. As the purpose of the Child Pornography Prevention Act is to protect children, a question may arise as to whether it is subject to punishment in the case where the image cannot be identified as that of a specific child. The idea is that the problem lies not in viewing or possessing child pornography, but in the fact that the subject is being victimized.

In Japan, depictions that sexually portray children in anime or manga are not subject to punishment. If generative AI creates a fictional character, it is expected to be treated the same way, regardless of how realistic it is. If it is an uncensored nude image or video, it will fall under the distribution of obscene objects under Article 175 of the Penal Code. However, if this is not the case, it is unclear what will happen as no cases have yet been prosecuted.

Meanwhile, when we look around the world, many countries prohibit child pornography even if the characters are fictional. In Europe, where sex crimes against children were frequent in the past, regulations are strict. Some countries even prohibit the production of pornography featuring adult women who look like children. In the face of the global trend of curbing organized crime, Japan began regulating child pornography. Given the international pressure, it is quite conceivable that Japan’s regulations will become stricter in the future.

How should we deal with generative AI? You are free to create your own. Showing it to someone, distributing it publicly, or sharing the information could violate laws. I believe that it is essential to pause to think before spreading such content, for the sake of both ourselves and society.

* The information contained herein is current as of January 2025.
* The contents of articles on Meiji.net are based on the personal ideas and opinions of the author and do not indicate the official opinion of Meiji University.
* I work to achieve SDGs related to the educational and research themes that I am currently engaged in.

Information noted in the articles and videos, such as positions and affiliations, are current at the time of production.