The AI Renaissance portrait generator isn’t great at painting people of color

Surprise! Artificial intelligence-generated portraits based off artwork from 15th century Europe… kind of suck at depicting people of color. 

Because we’re apparently always ready to hand over our photos for the sake of a trend, the internet’s current obsession is an AI portrait generator that deconstructs your selfies and rebuilds them as Renaissance and Baroque portraits. 

Created by researchers at the MIT-IBM Watson AI Lab, AI Portrait Ars is a fun way to see how you would have been perceived if you lived in another time period. 

“Portraits interpret the external beauty, social status, and then go beyond our body and face,” its creators wrote in the site’s “Why” section. “A portrait becomes a psychological analysis and a deep reflection on our existence.”

Unless, apparently, you’re not white. 

When I uploaded my photo into AI Portrait Ars, I was both horrified and amused by what the tool spat out. The painted version of me had my dark circles and eyebrows — at least my morning makeup routine is pulling through — but it whitened my skin to an unearthly pale tone, turned my flat nose into one with a prominent bridge and pointed end, and replaced my very hooded eyes with heavily lidded ones. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: MORGAN SUNG / AI PORTRAIT AR

We won’t even talk about the atrocity that was supposed to be my lips. 

The researchers wrote that the AI pulls from a data set of “tens of thousands of paintings from the Early Renaissance to Contemporary Art” using Generative Adversarial Network (GAN) models to produce “different styles and levels of abstraction.” 

There are two neural networks at play here: a discriminator, which figures out what makes a face and can recognize portraits, and a generator, which paints the portraits. Unlike neural style transfers, which take images and alter them with unique colors and textures while preserving the original’s features, the AI Portrait Ars “paintings” completely redesign the original subject. In an example provided by the site, the creators say that the model settled on a “Renaissance style” and recreated its (white) subject by “highlighting the elegance of the aquiline nose, the smoothness of the forehead.” 

“This type of portraiture is quite distinctive of the Western artistic tradition,” the creators continued on the site’s “Why” section. “Training our models on a data set with such a strong bias leads us to reflect on the importance of AI fairness.”

The creators at the MIT-IBM Watson AI Lab didn’t respond to request for comment, and it’s unclear how far the data set of artwork extends. Although the site says the data set includes contemporary art, it doesn’t say whether it includes art featuring subjects of color. 

The creators also emphasize the fact that the AI is focused on 15th century art, which could explain all the straight, high noses and much paler skin tones.

Even when the skin color change wasn’t as drastic, the AI does appear to whitewash distinct features. Coin Center’s Director of Communications Neeraj Agrawal said he was surprised at how his portrait turned out. 

“The nose threw me off,” he told Mashable in a Twitter DM. “The guy looked more Italian than Indian.”

Polygon reporter Petrana Radulovic pointed out that the AI lightened her eyes and gave her nose a much higher bridge.

And in a version that didn’t lighten my skin tone, the AI still gave me a much narrower, sharper nose than my own. 

As the AI Portrait AR creators note, the bias also extends to open-mouth smiles. Renaissance portrait artists rarely depicted their subjects with a toothy grin, because that feature was associated with a more comic painting genre — so the AI tends to swap out large smiles for the tight-lipped grimaces from its data set. 

“The inability of artificial intelligence to reproduce our smiles is teaching us something about the history of art,” the site’s description says. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: MORGAN SUNG / AI PORTRAIT AR

That’s not to say that there aren’t any examples of non-white portrait subjects — the Tumblr account medievalpoc highlights plenty of people of color throughout European art history. While the artwork that can fit into the definition of “portrait” is limited, there are definitely examples of merchants, nobility, and clergymen who certainly weren’t white. 

Ka Bradley, a British Cambodian Twitter user, acknowledged that there could be cultural blindspots — Islamic laws forbade the depiction of sentient beings in artwork, for example. But even then, she challenges why European art is considered “classical” while art from other parts of the world during the same period isn’t.

“I guess it’s partly to do with Western Europe being the dominant cultural hegemony, with a history of colonizing other lands,” she said in a Twitter DM to Mashable. “So Western portraiture is ‘classical art’ … whereas figurative/portrait art from other spheres [around] the same time is [an] ‘artifact’ or ‘ruin,’ separated from its own developmental timeline and understood through the lens of the colonizer.”

That being said, she wasn’t particularly happy with her AI portrait, either. 

Want some more examples? Here are some celebrities of color who had some, uh, especially strange results with the portrait generator. 

BTS‘s Jungkook was given more rounded brows, and his monolids were replaced with heavily lidded eyes.

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

While it kept her hair color, the AI narrowed Kehlani’s nose and lightened her skin. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

The generator also lightened Tessa Thompson‘s skin color, and turned her braided hair into some kind of bulky hat. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

Dwayne “The Rock” Johnson went from, well, “The Rock”… to a light pink thumb with green eyes. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

There are no words to describe what it did to Lizzo.

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

For some celebrities, instead of altering skin tones, the AI resorted to depicting their darker shades as sketches. It still gave Idris Elba a prominent, European bridge and deeper-set eyes, though. 

And it turned Jason Momoa into a hurried sketch of a grizzled ship captain, which may actually be pretty appropriate. 

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

The AI Renaissance portrait generator isn't great at painting people of color

Image: GETTY IMAGES / AI PORTRAIT AR

As its creators acknowledge, the AI Portrait AR is an experiment in exploring the bias of the model. That being said, it raises eyebrows when the model is better at portraying a meme than an actual human being who doesn’t happen to be white. 

It’s reminiscent of Google’s Arts and Culture app, which promised to match users’ selfies to documented works of art. Although it was well meaning, the app’s database didn’t include much artwork from non-European or American artists, and it struggled to match the faces of people of color. 

The creators of AI Portrait Ars do nod to the vibrant portraiture of non-European cultures, but as they acknowledge, the software was trained on a data set consisting of mostly white features, and so it still can’t identify and generate physical features that people of color have long been ostracized and othered for having.

Before social media helped normalize them, Western society has long looked down upon broad noses, darker skin tones, larger lips, and many other features. To overlook the depiction of people of color in Western art history, and omit them from the data set used to train a tool like this, isn’t just a blind spot, but literal erasure.

While the AI Portrait AR is a fun example of how powerful technology has become, it’s still disappointing for whole swaths of other cultures’ physical features to be excluded from “art.”

Cms%252f2019%252f7%252f2ed4d1dc 53e8 7f4e%252fthumb%252f00001.jpg%252foriginal.jpg?signature=l wpp05fkbjstuxksxslup 7kyy=&source=https%3a%2f%2fvdist.aws.mashable


Source

more recommended stories

  • DoorDash will finally fix its controversial tipping structure

    Food delivery service DoorDash has been.

  • Samsung Galaxy Note 10+ review: Beyond big

    With so many features and ways.

  • Amazon’s Cloud Cam is on sale for 25% off — its lowest price ever

    Just to let you know, if.

  • Facebook wants to hire journalists, but where does that leave News Feed?

    Facebook’s news plans are starting to.

  • Study finds connection between using emoji and having sex

    Recent studies suggest that frequent use.

  • Huawei’s foldable phone may come with a more powerful chip and camera

    Huawei’s Mate X — the company’s.

  • Finally a ‘dream job’ contest that let’s you decide what that is

    Follow your dreams.  That’s nice advice.

  • Making calls with Siri or Google could lead to you getting scammed

    If you’ve ever used Siri or.

  • Flip flops made of grass, live hermit crabs, and more weird stuff on Amazon this week

    Just to let you know, if.

  • Netflix’s ‘Diagnosis’ is real-life ‘House’ for the digital age: Review

    The following is a spoiler-free review.

  • TomTom GPS devices on sale for 30% off at Amazon — now $118

    Just to let you know, if.

  • Bumble and Tinder are paying frats to throw parties, acquire new users

    The wholesome, all-American tradition of drunken.

  • Jonathan Groff sings insanely sweet voice memos for kids as Kristoff from ‘Frozen’

    RIP my ovaries: Jonathan Groff, who.

  • Samsung might launch a phone with a graphene battery next year

    Samsung is working to launch a.

  • Bear breaks into a house and escapes ‘like the Kool-Aid Man’ when the cops arrive

    Bears are a lot like the.

  • Ninja accuses Twitch of pushing porn on his unused account

    Ninja has beef with Twitch, the.

  • He tried to prank the DMV. Then his vanity license plate backfired big time.

    Everyone hates parking tickets. Not everyone,.

  • For Season 3, ‘GLOW’ stays in Vegas while everyone changes

    It’s an old saying, and super.

  • Orphaned baby koala gets a tiny arm cast after falling from tree

    These adorable images of a baby.

  • 46% of social media users are ‘worn out’ by politics on social media

    We’re more than a year away.

  • ‘Game of Thrones’ creators are heading to Netflix

    Game of Thrones creators and showrunners.

  • Stephen Colbert catches out Fox News admitting Trump might be a racist

    After comparing America’s gun culture to.

  • Get the Fire TV Stick and Echo Dot together for $25 off on Amazon

    Just to let you know, if.

  • Best back-to-school sales this week: iPad, Bose, Keurig

    Heads up: All products featured here.

  • Cloudflare announces termination of 8chan’s service

    Cloudflare has announced that it will.

  • Pre-order ‘FIFA 20’ from GAME for under £35 (UK deal)

    Just to let you know, if.

  • Hailstones the size of baseballs pound parts of Canada

    The Earth continues to drop casual.

  • Domain registry moves to ban cryptocurrency names

    Sorry, crypto companies, you can’t register.

  • Desperate farmer destroys rare Lion King toy on live TV to make a point about online bullying

    On Friday morning, Australian time, a.

  • Soon you might be able to access Google Play content for $4.99 a month

    Disclosure Every product here is independently.

  • ‘Queer Eye’ stars share the disgusting incident that made Antoni literally pee himself laughing

    You’re used to the cast of.

  • No, you can’t watch Netflix while driving your Tesla

    While an upcoming update means you’ll.