With its avatar-generating AI, Lensa has been ascending the app store hit lists and turning off artists. It turns out that using the platform to create non-consensual soft porn is doable and far too simple, which is another reason to fly the flag.
TechCrunch has seen photo collections created with the Lensa app that contain pictures of women with bare breasts and nipples in pictures of people with recognizable faces. We made the decision to try it ourselves because it seemed like the kind of thing that shouldn’t have been possible. We made two sets of Lensa avatars to test whether Lensa would produce the images it might not be supposed to:
- Based on a single series of 15 images of a well-known actor.
- A second set was created using the same 15 photographs, but this time, five extra photos of the same actor’s face were added and Photoshop onto topless models.
The initial round of pictures matched the AI avatars that Lensa AI had previously created. But the second set was much hotter than we anticipated. The AI appears to disengage an NSFW filter after interpreting the Photoshop images as a license to behave however it pleases. 11 of the 100 images were topless shots, 11 of which had superior quality (or, at the very least, greater aesthetic consistency) than the subparly altered topless photos used as input to the AI.
It’s one thing to create lewd pictures of celebrities, and as shown by the sources we were able to locate, there have always been people eager to combine certain photographs in Photoshop. It doesn’t make it okay just because it’s widespread; in fact, famous people deserve their privacy and shouldn’t be subjected to non-consensual sexualized images. But so far, using photo editing software and putting in hours, if not days, of labor is required to make things appear realistic.
The major turning point—and ethical nightmare—is how simple it is to produce hundreds of nearly photo realistic AI-generated art works using nothing more than a smartphone, an app, and a few dollars.
It’s frightening how quickly you can generate images of anyone you can think of (or, at the very least, anyone you have a few pictures of). When NSFW content is included, things get a lot murkier rapidly. Your pals or some random person you met in a bar and added to your Facebook friend list might not have given their okay to someone making softcore pornography of them.
It seems that Lensa will happily produce a number of problematic images provided you have 10-15 “genuine” shots of a person and are prepared to spend the time photoshoping a few fakes.
Numerous pornographic images are already being produced by AI art generators; examples are Unstable Diffusion and others. The U.K. government is pushing for regulations that make the distribution of non-consensual nude pictures illegal because these platforms and the unchecked growth of other so-called “deepfake” platforms are developing into an ethical nightmare. This seems like a really fantastic idea, but the internet is already difficult to regulate, and we all have to deal with a mountain of moral, legal, and ethical dilemmas.
