May 19, 2016

A Dartmouth College competition has found that robots would be starving artists if they tried to write literature or poetry, although they may fare better as dance music DJ’s.

Organizers of Dartmouth’s “Turing Tests in Creative Arts” announced last night there were no winners in their competition for artificial intelligence algorithms that tried to create “human-quality” short stories, sonnets and dance music sets. The goal was to show whether human judges – in the form of literary readers and party-goers on a dance floor – can distinguish which creations were generated by machines and which by humans.

The contest’s organizers expected dozens of entries in each category but received far fewer submissions, a testament to the difficulty of writing sophisticated code that creates another dimension of AI — creative intelligence.

In the poetry contest, there were two entries that each generated two sonnets; the judges also received six sonnets written by humans. In the literature contest, there were three entries that each generated a short story; the judges also received eight short stories written by humans.

Passing the Turing Test required that a majority of the judges rate the entries as “human.” None of the machine-generated sonnets or short stories fooled any of the judges except one short story that fooled one judge.

“Our algorithms seem not yet able to imitate human kinds of poetry, but the code that was submitted was still amazing,” says Dan Rockmore, a professor of mathematics and computer science, who organized the contest along with Michael Casey, a professor of music and computer science.

For the dance music sets, which were either music selected by human DJ’s or by machines, there were eight algorithm entries. Two of the entries fooled about 40 percent of human voters who were on the dance floor.

Although there were no winners, the contest organizers said several of the entries were sophisticated enough to deserve a portion of the prize money.

The Turing test, named for the British pioneering computer scientist and mathematician Alan Turing, tests a machine’s ability to exhibit intelligent behavior that is indistinguishable from that of a human.

Full contest results and an example of one of the machine-generated sonnets are available at http://bregman.dartmouth.edu/turingtests/

Available to comment are Professor Dan Rockmore at daniel.n.rockmore@dartmouth.edu and Professor Michael Casey at michael.a.casey@dartmouth.edu.