Message 00106 [Homepage] [Navigation]
Thread: joxT00000 Message: 77/176 L20 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: [jox] Re: Peer Review

Dear Mathieu, CSPP Board Members, all!

Last week (9 days ago) Mathieu O'Neil wrote:
OK, I contacted one of the authors of the FM paper and he kindly
sent me links to his second part as well as another relevant

Mathieu sent the links privately. Indeed a very useful paper. I'll
respond to this here because otherwise the communication becomes to
confusing for me ;-) .

In his mail he also mentioned that they are seeking feedback from
people on these issues. I responded that would be possible but that
I was acting as part of a group and queried whether it would be OK
to circulate the papers to this group (without really saying who was
in it though I had originally sent a link to this list's archive). 

They are welcome. But then we need the right to put the link here
(it's a public link anyway so this should not matter too much).

@Mathieu: I hope you won't mind that I quote the following questions
from your private mail here.

Last week (7 days ago) Mathieu O'Neil wrote:
I guess the main questions are: which features would we want?

In general I think those guys are very much on the same track as we
:-) . I think the key question to solve I already mentioned in my post
from `Mon, 24 Aug 2009 12:25:45 +0200`__. To prevent leaking too much
information from the paper here I only reply to what is new or very
different from my suggestions there or comments from others made here
already. I relate to the chapter "DEMOCRATIC ONLINE KNOWLEDGE


* Multi-level ratings

  Similar to my previous suggestion the submissions can and should be
  given a number of levels reflecting how the reviewers / editorial
  board assesses the submission. That solves the problem of the binary
  accept / reject scheme. On a website it is also easy to flag
  articles appropriately.

  Also using several rating dimensions is a good idea. What is needed,
  however, is to reduce the many dimensions to a single number to give
  orientation to readers. May be the scientific vs. activist category
  could be one such dimension.

  I think the reviewers should still have the responsibility to rate
  an article. May be the reader rating could be one more dimension -
  given that there are enough ratings from the readers.

  Readers need to have an option to tune what dimensions they consider
  most important. Technically this would mean to have different
  reports using a reader's weights. That is technically simple.

* Submissions

  I think every submission should be addressed to the editorial board.
  IMHO this also adds a level of responsibility on the part of the
  submitters: It's a difference whether you just drop something in an
  electronic system or ask real people to consider your submission.

  This doesn't mean that a submission is withheld from the website. It
  can be put there immediately if the author wishes so.

* Anonymous contributions

  In general I don't like anonymous contributions as well as I don't
  like near-anonymous contributions from obvious pseudonyms. I see
  that creation of an account actually feels like an obstacle - at
  least to me. But nonetheless I don't like anonymous contributions.
  IMHO this at least reduces the spam problem very much.

Which of those
we want can we have without too much hassle?

I think most if not all of their suggestions and our ideas are
technically feasible. IMHO the real work (i.e. human labor) is reading
submissions, thinking about them and making decisions. This real work
needs to be done anyway. The rest can and should be left to the



Thread: joxT00000 Message: 77/176 L20 [In date index] [In thread index]
Message 00106 [Homepage] [Navigation]