Message 00156 [Homepage] [Navigation]
Thread: joxT00000 Message: 109/176 L36 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: Multi-rating mode of evaluation / Updating papers (was: Re: [jox] Pot-pourri)

Why would editorial rating allow us to publish more papers? Since this is 
not a paper publication, we can publish as many as we want anyway. 
Editorial rating would allow us to publish some papers, but grade them 
poorly, flagging them as 'barely good enough'. We shouldn't do this. We 
should publish only papers that we agree are fit for publication. 

Reader ratings are different. That might useful form of feedback.

As for changes, I rather give authors a possibility to reply to the 
feedback they get after publication. It's more transparent. If they want to 
publish a substantially new version, they can do so as a new paper.

I think one of the things a journal is really valuable for is not just 
putting things out, but providing long-term stability to a publication and 
thus serve also a record of the evolution of thought.


On Saturday, 21. November 2009, Mathieu O'Neil wrote:
[Converted from multipart/alternative]

[1 text/plain]
Hi all

Thanks to everyone who has contributed to the process this week. I just
 wanted to alert everyone that there is an issue that needs to be
 resolved: Felix and Athina (see below) have objected to our stated
 policy of "rating" submissions according to a scale of -1 to +5
 (inspired by Brian Whitworth and Rob Friedman's papers on academic
 publishing in First Monday). As you know, this would in theory
 facilitate the publication of a greater number of submissions than is
 the case in other journals (though if you consider the fact that some
 academic journals publish 5 or 6 issues each year, that represents a
 fair number of papers).

We could perhaps simplify the rating system but this would not change
 the fact that there would be a situation where some published
 submissions are given a higher "expert rating" than others before
 publication (as well as a "reader rating" given by registered users
 after publication). The question of how people who have been given an
 inferior rating by reviewers would feel about this situation has not
 been addressed so far and I think it needs to be. If we do not rate
 then do we go to (in theory) a lower publication rate where only
 excellent contributions are published?

I urge everyone to express their opionion about this core mechanism in
 our peer review process...


ps. Another important issue was raised by Brian (see below) which we had
 discussed previously on this list: whether authors can update their
 papers after receiving reader feedback. As I thought more about this I
 realised that having version numbers (ie v.4.8) might correct the
 problem of not knowing which version to refer to. So I'm starting to
 reexamine my position about this. I can think of several online texts
 which had several versions, Wark's The Hacker Manifesto, Raymond's
 Cathedral and the Bazaar, and I know that when I wrote an article on
 Wikipedia for a magazine this year, the English version is more
 accurate than the original French because I got some feedback from the
 French - and then I got more feedback on the English version...  I
 think it might be a good thing as long as its not constant (say you
 could only create a revised version a limited number of times a year?)

(Felix Stalder: )

As for the review process, I think having people to be able to
comment is
great. I'm not so sure, if we should adopt a rating of published
from +2 to +5. This would mean that the reviewers tell the
audience which
papers they think are every good, and which are merely good
enough. I think
this is a bit unfair to the published authors and creates biases
in the
audience. After all, wouldn't you start reading papers that are
high, rather than those ranked poorly?

(Athina Karatzogianni:)

in global politics. I agree with Felix, I never thought that the
rating was
a good idea myself.


On Tue, Nov 17, 2009 at 10:33 PM, Mathieu O'Neil

[Converted from multipart/alternative]

[1 text/plain]
Hi all

=Brian Whitworth=

This is good news, and I am happy to assist in any way I can.

Our main idea

is to go from a static KES to a dynamic one, e.g. authors can update
versions of what they write based on reader feedback, so

things evolve

rather than being fixed in stone, plus with attribution. When

the changes

stabilize, one can still do fixed print. So most current

publishing manager

software design's wont cut it, like manuscript central, as a

secret reviewer

feedback system is behind the publication "shop window". You

can still have

that right at the beginning, by author choice, but the main

idea is a system

tht is transparent and open. If you can make a move in that

direction that

works, or implement the design the second part of our paper

outlined in any

way, I encourage you.


Thanks for your message, and for your offer to help. I will

put you down as

part of our scientific committee. Regarding the updating of

content, we

discussed this, and came to the conclusion that while we are

all for readers

leaving comments to papers, and authors responding to readers

in comments,

the notion of a permanently mutable text (as is the case for

example on WP)

did not really suit us in the context of a journal. I can see

the attraction

in what you suggest below but it does raise a number of issues:
- at what point is an article "stable", if the possibility

always exists

that some new argument, for or against, comes along and must

be dealt

- another possible problem with continuously changing the main

part of a

paper is that it would create uncertainty as to which version

was being

referred to in subsequent publications.
So we decided that we would rather have people write a new

paper based on

all the comments and responses, if necessary. At the same time

this is

something that we can talk about more. I will subscribe you to

our list and

raise this alongside other issues that come up.

[2 text/html]


--- ----------------------------- out now:
*|Mediale Kunst/Media Arts Zurich.13 Positions.Scheidegger&Spiess2008
*|Manuel Castells and the Theory of the Network Society. Polity, 2006 
*|Open Cultures and the Nature of Networks. Ed. Futura/Revolver, 2005 

Thread: joxT00000 Message: 109/176 L36 [In date index] [In thread index]
Message 00156 [Homepage] [Navigation]