Message 00165 [Homepage] [Navigation]
Thread: joxT00000 Message: 117/176 L37 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: [jox] Re: Multi-rating mode of evaluation / Updating papers

Hi all,

Stefan Meretz wrote:
The journal might contain strong pov articles, which is good. One aim of 
the Journal must be to support a "thinking against the mainstream". If 
peer production is really a new thing, then theorzing around this topic 
will and has to be new and unfamiliar. -- Will the expert committee work 
in this fashion? Or are unfamiliar povs are rated out, because the pov 
is not shared? (this does say anything against the persons listed which 
I don't know).

if a public rating system is used, POV and other controversial articles will
generally end up with an average rating, since some reviewers like them,
while others don't. Hence it might indeed be better to stick with a binary
"publish / don't publish" decision from the journal editors / reviewing
committee, and let the readers do the ratings. Of course, reader ratings
will have the same effect of punishing controversial items, but at least
reader ratings don't sound quite as "official" as committee ratings.
(Incidentally, I noted that effect with the IMDB: excellent, but unusual
movies often get an average rating -- typically, high ratings do indeed
indicate that a movie is good, but average or poor rating don't give much
reliable information about the movie.)

Of course, internally the reviewers will probably use some kind of rating
system, since peer review usually does, and if we want transparency we can't
hide these ratings from the readers -- but at least I wouldn't show them by
default, but place them somewhere in the background information about the
article (reader has to click on "reviewing process / reviewer feedback" or
something like that).

Best regards

|------- Dr. Christian Siefkes ------- christian -------
| Homepage: | Blog:
|    Peer Production Everywhere:
|---------------------------------- OpenPGP Key ID: 0x346452D8 --
A misleading benchmark test can accomplish in minutes what years of good
engineering can never do.
        -- Dilbert

Thread: joxT00000 Message: 117/176 L37 [In date index] [In thread index]
Message 00165 [Homepage] [Navigation]