Dartmouth College Partners with the Center for Open Science and the Center for Political Studies to Launch New Research Contest to Study the 2016 Election


September 8, 2016 — Dartmouth College has partnered with the Center for Open Science and the Center for Political Studies at the University of Michigan’s Institute for Social Research to launch the Election Research Preacceptance Competition— an innovative initiative that will test a new approach to conducting and publishing political science research during the 2016 election. The competition provides an opportunity for researchers to help citizens understand why Americans voted the way they did while also strengthening the evidence base of political science and election-oriented public opinion research.

Academic publishing typically relies on peer review to assess the quality of the research after it is conducted, which can lead to a bias toward unexpected, statistically significant results. The competition aims to increase the credibility of political science by fielding proposals for articles and vetting them through a peer review process before the results are known. This new approach helps focus the review process on the importance of the research question and the quality of the methodology used to answer it rather than the results alone.

“Preacceptance allows studies to be reviewed based on the theoretical contribution and the quality of the research design rather than whether the author’s expectations were supported,” says Brendan Nyhan, a professor of government at Dartmouth College and a co-organizer of the competition, who has studied the credibility of political science research.

Entrants in the competition will preregister a research design intended to study an important aspect of the 2016 general election using data collected by the American National Election Studies (ANES). A condition of entering the competition is that entrants must complete and register a design before the ANES data are released. Many leading academic journals have agreed to review scholarly articles that include these research plans and to review them before the data are available or results are known.

“This competition offers $2,000 prizes and speaking opportunities at major academic conferences to encourage scholars to engage in new and innovative research about a critical event in our country’s history,” says Arthur Lupia, Hal R. Varian Collegiate Professor of Political Science at the University of Michigan and a co-organizer of the competition.

The competition brings together many of the field’s major journals in an unprecedented cooperation. These journals have also agreed to consider sharing reviews, which can help scholars publish important results more quickly.

The competition is funded by a grant from the Laura and John Arnold Foundation, which is supporting similar work including the Center for Open Science’s Preregistration Challenge, a broad research competition that rewards researchers from any field for publishing the results of preregistered research.

“We built a workflow to guide researchers as they create their analysis plans, which are otherwise typically not specified until after data collection,” says David Mellor, Project Manager from the Center for Open Science, which builds the free and open source Open Science Framework. “For this competition, researchers fill out a couple of questions and their analysis plans are archived and time stamped, enabling them to specify their hypotheses and analysis plans in advance and thereby ensuring the rigor of the scientific inquiry process.”

Available for comment are: Brendan Nyhan at Dartmouth (brendan.j.nyhan@dartmouth.edu); David Mellor at the Center for Open Science (david@cos.io); and Arthur Lupia at the Center for Political Studies at the University of Michigan’s Institute for Social Research (lupia@umich.edu).