Can Robot Artists Create Human-Quality Work? Not Yet

News subtitle

Robots would be starving artists if they tried to write literature or poetry.

Image
Image
Turing Tests in Creative Arts
The goal of the Turing Tests was to show whether human judges—in the form of literary readers and partygoers on a dance floor— could distinguish which creations were generated by machines and which by humans. (Photo by Joshua Renaud ’17)
Body

Robots would be starving artists if they tried to write literature or poetry, although they may fare better as dance music DJ’s.

That’s the conclusion of the human judges in Dartmouth’s “Turing test” competition, which took place this week.

Organizers of the “Turing Tests in Creative Arts” announced Wednesday night there were no winners in their competition for artificial intelligence algorithms that tried to create “human-quality” short stories, sonnets, and dance music sets. The goal was to show whether human judges—in the form of literary readers and partygoers on a dance floor— could distinguish which creations were generated by machines and which by humans.

“Our algorithms seem not yet able to imitate human kinds of poetry, but the code that was submitted was still amazing,” says Daniel Rockmore, a professor of mathematics and the William H. Neukom 1964 Distinguished Professor of Computational Science, who organized the contest along with Michael Casey, a professor of computer science and the James Wright Professor of Music.

Rockmore and Casey had expected dozens of entries in each category but received far fewer submissions, which they say is a testament to the difficulty of writing sophisticated code that creates another dimension of AI—creative intelligence.

In the poetry contest, there were two entries that each generated two sonnets; the judges also received six sonnets written by humans. In the literature contest, there were three entries that each generated a short story; the judges also received eight short stories written by humans.

Passing the Turing Test required that a majority of the judges rate the entries as “human.” None of the machine-generated sonnets or short stories fooled any of the judges with the exception of one short story that fooled one judge.

For the dance music sets, which were music selected by either human DJs or machines, there were eight algorithm entries. Two of the entries fooled about 40 percent of human voters who were on the dance floor.

Although there were no winners, the contest organizers said several of the entries in each category were sophisticated enough to deserve a portion of the prize money.

The Turing test, named for the British pioneering computer scientist and mathematician Alan Turing, tests a machine’s ability to exhibit intelligent behavior that is indistinguishable from that of a human.

See full contest results and an example of one of the machine-generated sonnets.

John Cramer