[Marxism] Web as alternative to peer review

Louis Proyect lnp3 at panix.com
Tue Aug 24 12:30:37 MDT 2010


(I haven't submitted anything to a peer-reviewed journal in over 5 
years at least. Frankly, I find the entire ritual more off-putting 
than a prostate exam. Thank god I am not in academia where I would 
have to put up that--peer reviews, that is, not prostate exams.)

NY Times August 23, 2010
Scholars Test Web Alternative to Peer Review
By PATRICIA COHEN

For professors, publishing in elite journals is an unavoidable 
part of university life. The grueling process of subjecting work 
to the up-or-down judgment of credentialed scholarly peers has 
been a cornerstone of academic culture since at least the mid-20th 
century.

Now some humanities scholars have begun to challenge the monopoly 
that peer review has on admission to career-making journals and, 
as a consequence, to the charmed circle of tenured academe. They 
argue that in an era of digital media there is a better way to 
assess the quality of work. Instead of relying on a few experts 
selected by leading publications, they advocate using the Internet 
to expose scholarly thinking to the swift collective judgment of a 
much broader interested audience.

“What we’re experiencing now is the most important transformation 
in our reading and writing tools since the invention of movable 
type,” said Katherine Rowe, a Renaissance specialist and media 
historian at Bryn Mawr College. “The way scholarly exchange is 
moving is radical, and we need to think about what it means for 
our fields.”

That transformation was behind the recent decision by the 
prestigious 60-year-old Shakespeare Quarterly to embark on an 
uncharacteristic experiment in the forthcoming fall issue — one 
that will make it, Ms. Rowe says, the first traditional humanities 
journal to open its reviewing to the World Wide Web.

Mixing traditional and new methods, the journal posted online four 
essays not yet accepted for publication, and a core group of 
experts — what Ms. Rowe called “our crowd sourcing” — were invited 
to post their signed comments on the Web site MediaCommons, a 
scholarly digital network. Others could add their thoughts as 
well, after registering with their own names. In the end 41 people 
made more than 350 comments, many of which elicited responses from 
the authors. The revised essays were then reviewed by the 
quarterly’s editors, who made the final decision to include them 
in the printed journal, due out Sept. 17.

The Shakespeare Quarterly trial, along with a handful of other 
trailblazing digital experiments, goes to the very nature of the 
scholarly enterprise. Traditional peer review has shaped the way 
new research has been screened for quality and then how it is 
communicated; it has defined the border between the public and an 
exclusive group of specialized experts.

Today a small vanguard of digitally adept scholars is rethinking 
how knowledge is understood and judged by inviting online readers 
to comment on books in progress, compiling journals from blog 
posts and sometimes successfully petitioning their universities to 
grant promotions and tenure on the basis of non-peer-reviewed 
projects.

The quarterly’s experiment has so far inspired at least one other 
journal — Postmedieval — to plan a similar trial for next year.

Just a few years ago these sorts of developments would have been 
unthinkable, said Dan Cohen, director of the Center for History 
and New Media at George Mason University. “Serious scholars are 
asking whether the institutions of the academy — as they have 
existed for decades, even centuries — aren’t becoming obsolete,” 
he said.

Ms. Rowe, who served as guest editor for The Shakespeare 
Quarterly’s special issue devoted to Shakespeare and new media, 
said: “The traditional process is not so much a gold standard but 
an effective accommodation to the needs of the field. It 
represents a settlement for a particular moment, not a perfect ideal.”

Each type of review has benefits and drawbacks.

The traditional method, in which independent experts evaluate a 
submission, often under a veil of anonymity, can take months, even 
years.

Clubby exclusiveness, sloppy editing and fraud have all marred 
peer review on occasion. Anonymity can help prevent personal bias, 
but it can also make reviewers less accountable; exclusiveness can 
help ensure quality control but can also narrow the range of 
feedback and participants. Open review more closely resembles 
Wikipedia behind the scenes, where anyone with an interest can 
post a comment. This open-door policy has made Wikipedia, on 
balance, a crucial reference resource.

Ms. Rowe said the goal is not necessarily to replace peer review 
but to use other, more open methods as well.

In some respects scientists and economists who have created online 
repositories for unpublished working papers, like repec.org, have 
more quickly adapted to digital life. Just this month, 
mathematicians used blogs and wikis to evaluate a supposed 
mathematical proof in the space of a week — the scholarly 
equivalent of warp speed.

In the humanities, in which the monograph has been king, there is 
more inertia. “We have never done it that way before,” should be 
academia’s motto, said Kathleen Fitzpatrick, a professor of media 
studies at Pomona College.

Ms. Fitzpatrick was a founder of the MediaCommons network in 2007. 
She posted chapters of her own book “Planned Obsolescence” on the 
site, and she used the comments readers provided to revise the 
manuscript for NYU Press. She also included the project in the 
package she presented to the committee that promoted her to full 
professor this year.

Many professors, of course, are wary of turning peer review into 
an “American Idol”-like competition. They question whether people 
would be as frank in public, and they worry that comments would be 
short and episodic, rather than comprehensive and conceptual, and 
that know-nothings would predominate.

After all, the development of peer review was an outgrowth of the 
professionalization of disciplines from mathematics to history — a 
way of keeping eager but uninformed amateurs out.

“Knowledge is not democratic,” said Michèle Lamont, a Harvard 
sociologist who analyzes peer review in her 2009 book, “How 
Professors Think: Inside the Curious World of Academic Judgment.” 
Evaluating originality and intellectual significance, she said, 
can be done only by those who are expert in a field.

At the same time she noted that the Web is already having an 
incalculable effect on academia, especially among younger 
professors. In her own discipline, for instance, the debates 
happening on the site Sociologica.mulino.it “are defined as being 
frontier knowledge even though they are not peer reviewed.”

The most daunting obstacle to opening up the process is that 
peer-review publishing is the path to a job and tenure, and no 
would-be professor wants to be the academic canary in the coal mine.

The first question that Alan Galey, a junior faculty member at the 
University of Toronto, asked when deciding to participate in The 
Shakespeare Quarterly’s experiment was whether his essay would 
ultimately count toward tenure. “I went straight to the dean with 
it,” Mr. Galey said. (It would.)

Although initially cautious, Mr. Galey said he is now “entirely 
won over by the open peer review model.” The comments were more 
extensive and more insightful, he said, than he otherwise would 
have received on his essay, which discusses Shakespeare in the 
context of information theory.

Advocates of more open reviewing, like Mr. Cohen at George Mason 
argue that other important scholarly values besides quality 
control — for example, generating discussion, improving works in 
progress and sharing information rapidly — are given short shrift 
under the current system.

“There is an ethical imperative to share information,” said Mr. 
Cohen, who regularly posts his work online, where he said 
thousands read it. Engaging people in different disciplines and 
from outside academia has made his scholarship better, he said.

To Mr. Cohen, the most pressing intellectual issue in the next 
decade is this tension between the insular, specialized world of 
expert scholarship and the open and free-wheeling exchange of 
information on the Web. “And academia,” he said, “is caught in the 
middle.”




More information about the Marxism mailing list