Message 00156 [Homepage] [Navigation]
Thread: joxT00000 Message: 109/176 L36 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: Multi-rating mode of evaluation / Updating papers (was: Re: [jox] Pot-pourri)

Why would editorial rating allow us to publish more papers? Since this is 
not a paper publication, we can publish as many as we want anyway. 
Editorial rating would allow us to publish some papers, but grade them 
poorly, flagging them as 'barely good enough'. We shouldn't do this. We 
should publish only papers that we agree are fit for publication. 

Reader ratings are different. That might useful form of feedback.

As for changes, I rather give authors a possibility to reply to the 
feedback they get after publication. It's more transparent. If they want to 
publish a substantially new version, they can do so as a new paper.

I think one of the things a journal is really valuable for is not just 
putting things out, but providing long-term stability to a publication and 
thus serve also a record of the evolution of thought.

Felix



On Saturday, 21. November 2009, Mathieu O'Neil wrote:

[Converted from multipart/alternative]

[1 text/plain]
Hi all

Thanks to everyone who has contributed to the process this week. I just
 wanted to alert everyone that there is an issue that needs to be
 resolved: Felix and Athina (see below) have objected to our stated
 policy of "rating" submissions according to a scale of -1 to +5
 (inspired by Brian Whitworth and Rob Friedman's papers on academic
 publishing in First Monday). As you know, this would in theory
 facilitate the publication of a greater number of submissions than is
 the case in other journals (though if you consider the fact that some
 academic journals publish 5 or 6 issues each year, that represents a
 fair number of papers).

We could perhaps simplify the rating system but this would not change
 the fact that there would be a situation where some published
 submissions are given a higher "expert rating" than others before
 publication (as well as a "reader rating" given by registered users
 after publication). The question of how people who have been given an
 inferior rating by reviewers would feel about this situation has not
 been addressed so far and I think it needs to be. If we do not rate
 then do we go to (in theory) a lower publication rate where only
 excellent contributions are published?

I urge everyone to express their opionion about this core mechanism in
 our peer review process...

cheers,
Mathieu

ps. Another important issue was raised by Brian (see below) which we had
 discussed previously on this list: whether authors can update their
 papers after receiving reader feedback. As I thought more about this I
 realised that having version numbers (ie v.4.8) might correct the
 problem of not knowing which version to refer to. So I'm starting to
 reexamine my position about this. I can think of several online texts
 which had several versions, Wark's The Hacker Manifesto, Raymond's
 Cathedral and the Bazaar, and I know that when I wrote an article on
 Wikipedia for a magazine this year, the English version is more
 accurate than the original French because I got some feedback from the
 French - and then I got more feedback on the English version...  I
 think it might be a good thing as long as its not constant (say you
 could only create a revised version a limited number of times a year?)
 Thoughts?...

(Felix Stalder: )


As for the review process, I think having people to be able to
comment is
great. I'm not so sure, if we should adopt a rating of published
papers
from +2 to +5. This would mean that the reviewers tell the
audience which
papers they think are every good, and which are merely good
enough. I think
this is a bit unfair to the published authors and creates biases
in the
audience. After all, wouldn't you start reading papers that are
ranked
high, rather than those ranked poorly?


(Athina Karatzogianni:)


in global politics. I agree with Felix, I never thought that the
rating was
a good idea myself.

Cheers
Athina

On Tue, Nov 17, 2009 at 10:33 PM, Mathieu O'Neil

<mathieu.oneil anu.edu.au>wrote:

[Converted from multipart/alternative]

[1 text/plain]
Hi all

=Brian Whitworth=

This is good news, and I am happy to assist in any way I can.


Our main idea


is to go from a static KES to a dynamic one, e.g. authors can update
versions of what they write based on reader feedback, so


things evolve


rather than being fixed in stone, plus with attribution. When


the changes


stabilize, one can still do fixed print. So most current


publishing manager


software design's wont cut it, like manuscript central, as a


secret reviewer


feedback system is behind the publication "shop window". You


can still have


that right at the beginning, by author choice, but the main


idea is a system


tht is transparent and open. If you can make a move in that


direction that


works, or implement the design the second part of our paper


outlined in any


way, I encourage you.


(Mathieu:)

Thanks for your message, and for your offer to help. I will


put you down as


part of our scientific committee. Regarding the updating of


content, we


discussed this, and came to the conclusion that while we are


all for readers


leaving comments to papers, and authors responding to readers


in comments,


the notion of a permanently mutable text (as is the case for


example on WP)


did not really suit us in the context of a journal. I can see


the attraction


in what you suggest below but it does raise a number of issues:
- at what point is an article "stable", if the possibility


always exists


that some new argument, for or against, comes along and must


be dealt


with/integrated?
- another possible problem with continuously changing the main


part of a


paper is that it would create uncertainty as to which version


was being


referred to in subsequent publications.
So we decided that we would rather have people write a new


paper based on


all the comments and responses, if necessary. At the same time


this is


something that we can talk about more. I will subscribe you to


our list and


raise this alongside other issues that come up.


[2 text/html]
______________________________
http://www.oekonux.org/journal




-- 







--- http://felix.openflows.com ----------------------------- out now:
*|Mediale Kunst/Media Arts Zurich.13 Positions.Scheidegger&Spiess2008
*|Manuel Castells and the Theory of the Network Society. Polity, 2006 
*|Open Cultures and the Nature of Networks. Ed. Futura/Revolver, 2005 
______________________________
http://www.oekonux.org/journal
Thread: joxT00000 Message: 109/176 L36 [In date index] [In thread index]
Message 00156 [Homepage] [Navigation]