Re: [jox] Re: Peer Review
- From: Stefan Merten <smerten oekonux.de>
- Date: Mon, 07 Sep 2009 11:13:13 +0200
Dear Mathieu, CSPP Board Members, all!
Last week (9 days ago) Mathieu O'Neil wrote:
OK, I contacted one of the authors of the FM paper and he kindly
sent me links to his second part as well as another relevant
chapter.
Mathieu sent the links privately. Indeed a very useful paper. I'll
respond to this here because otherwise the communication becomes to
confusing for me ;-) .
In his mail he also mentioned that they are seeking feedback from
people on these issues. I responded that would be possible but that
I was acting as part of a group and queried whether it would be OK
to circulate the papers to this group (without really saying who was
in it though I had originally sent a link to this list's archive).
They are welcome. But then we need the right to put the link here
(it's a public link anyway so this should not matter too much).
@Mathieu: I hope you won't mind that I quote the following questions
from your private mail here.
Last week (7 days ago) Mathieu O'Neil wrote:
I guess the main questions are: which features would we want?
In general I think those guys are very much on the same track as we
:-) . I think the key question to solve I already mentioned in my post
from `Mon, 24 Aug 2009 12:25:45 +0200`__. To prevent leaking too much
information from the paper here I only reply to what is new or very
different from my suggestions there or comments from others made here
already. I relate to the chapter "DEMOCRATIC ONLINE KNOWLEDGE
EXCHANGE".
__ http://www.oekonux.org/journal/list/archive/msg00091.html
* Multi-level ratings
Similar to my previous suggestion the submissions can and should be
given a number of levels reflecting how the reviewers / editorial
board assesses the submission. That solves the problem of the binary
accept / reject scheme. On a website it is also easy to flag
articles appropriately.
Also using several rating dimensions is a good idea. What is needed,
however, is to reduce the many dimensions to a single number to give
orientation to readers. May be the scientific vs. activist category
could be one such dimension.
I think the reviewers should still have the responsibility to rate
an article. May be the reader rating could be one more dimension -
given that there are enough ratings from the readers.
Readers need to have an option to tune what dimensions they consider
most important. Technically this would mean to have different
reports using a reader's weights. That is technically simple.
* Submissions
I think every submission should be addressed to the editorial board.
IMHO this also adds a level of responsibility on the part of the
submitters: It's a difference whether you just drop something in an
electronic system or ask real people to consider your submission.
This doesn't mean that a submission is withheld from the website. It
can be put there immediately if the author wishes so.
* Anonymous contributions
In general I don't like anonymous contributions as well as I don't
like near-anonymous contributions from obvious pseudonyms. I see
that creation of an account actually feels like an obstacle - at
least to me. But nonetheless I don't like anonymous contributions.
IMHO this at least reduces the spam problem very much.
Which of those
we want can we have without too much hassle?
I think most if not all of their suggestions and our ideas are
technically feasible. IMHO the real work (i.e. human labor) is reading
submissions, thinking about them and making decisions. This real work
needs to be done anyway. The rest can and should be left to the
machine.
Grüße
Stefan
______________________________
http://www.oekonux.org/journal