Message 00158 [Homepage] [Navigation]
Thread: joxT00000 Message: 111/176 L38 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: [jox] Re: Multi-rating mode of evaluation / Updating papers

[Converted from multipart/alternative]

[1 text/plain]
Hi Biella, Felix, all

Biella: no, these points have not been raised. I think your suggestion about reviewers communicating to compare their opinions is excellent.

Felix, I originally agreed with your position: have feedback and author reaction in the comment. But this goes against the flow of reading arguments in order. So maybe there could be feedback and author reactions in the comments section and authors could add brackets or links in the text indicating comment x is relevant to passage y. So I am starting to think the advantages of updating articles outweigh the downsides. 

I agree with Biella that vetting potential modifications is a big concern if there turns out, after a while, to be a large number of articles to monitor, so that is why I would suggest limiting the number of times authors could potentially update their papers per year. Presumably the changes we would encourage would not be all that significant... if someone wanted to rewrite completely whole sections that poses a different problem... in any case I'm not even sure if the CMS can be set up to allows authors to access texts directly, or whether we would want that to happen, or how it would work - give them access rights over one paper, for a limited period of time? 

Regarding ratings, I will wait to see what others have to say before commenting further.



From: Gabriella Coleman 
Felix Stalder wrote:
Why would editorial rating allow us to publish more papers? 
Since this is not a paper publication, we can publish as many as 
we want anyway. Editorial rating would allow us to publish some 
papers, but grade them poorly, flagging them as 'barely good 
enough'. We shouldn't do this. We should publish only papers 
that we agree are fit for publication. 

I agree with this: even if there are many problems with 
traditional peer review, what I like is that once something is 
accepted, the journal stands by it fully and I think this is 
something we should do too.

Reader ratings are different. That might useful form of feedback.

As for changes, I rather give authors a possibility to reply to 
the feedback they get after publication. It's more transparent. 
If they want to publish a substantially new version, they can do 
so as a new paper.

This is an interesting idea and also reminds me of a question I 
have about how we might conduct our peer review. First, my 
apologies if this has already been raised. I have been trying to 
catch up with the list, have only skimmed the First Monday 
article on peer review and the like but have had a tough few 
weeks traveling and living between two cities and am not 
completely caught up.

I was also wondering if the peer reviewers will touch base with 
each other about their reader reports before sending them to the 
editor/author and using that conversation as an opportunity to 
reassess the reports. I have always found one of the strange 
things about reviews is how you can get a glowing review, a 
lukewarm one, and finally one that is downright nasty. A 
conversation between reviewers might be a good way to push 
against some of the shortcomings of an individual reviewing 
social science/humanities texts where there is a great deal of 
judgment along the axis of personal taste that goes into 
assessing the strength of an essay.

As per updating a paper: I see no reason why there can't be a 
new version, although we would also probably have to have some 
sort of vetting process and I do worry about overburdening the 
readers/members of the journal doing the reviews.

Again apologies if this has been raised,

Gabriella Coleman, Assistant Professor
Department of Media, Culture, & Communication
New York University
239 Greene St, 7th floor
NY NY 10003

Dr Mathieu O'Neil
Adjunct Research Fellow
Australian Demographic and Social Research Institute
College of Arts and Social Science
The Australian National University

E-mail: mathieu.oneil
Tel.: (61 02) 61 25 38 00
Mail: Coombs Building, 9
Canberra, ACT 0200 - AUSTRALIA

[2 text/html]

Thread: joxT00000 Message: 111/176 L38 [In date index] [In thread index]
Message 00158 [Homepage] [Navigation]