[Converted from multipart/alternative]
[1 text/plain]
Dear all
Apologies for slackness, I've been travelling and am currently
snowed under. I want to re-read those papers again before
commenting on Stefans poins. But: It does not make sense to be
discussing openly something which only a few have seen so I'm
taking it upon myself to post the references here. I never got
an answer from the authors to my question as to whether it would
be OK to distribute. In French there is a saying: "Qui ne dit
mot consent". To potential readers: please remember the second
part of the First Monday paper is still a draft so do not quote.
My biggest question is: they suggest publishing everything that
is submitted with ratings ie 5 excellent, 4 good, etc even -1
(dont publish) if the author wants. I think this is an
interesting concept but I also think this is potentially going
towards the "process" option we discussed before. Perhaps
articles rated 4 or 5 could periodically be gathered into a
"stable" release that could be advertised across the weboverse?
Anyway here are the refs:
Hi Mathieu Thanks for your enquiry and interest. Your
journal looks good. The social design of a KES is quite
complex, but you can see our first attempt to lay out the
details and spirit of the endeavor at
http://brianwhitworth.com/BWRF-FM-Part2.pdf In particular
note the section on privacy, where we allow reviewers to
reveal themselves after the review is over if they want
to. Also note it is still draft until it comes out in
FM. Some more detail on Socio-technical design
in general is also given in our Handbook of STS Design,
see http://brianwhitworth.com/sts for which mashup is a
first go at making such stuff available. In particular,
check out my chapter 1 which gives an idea of some of the
complexity, see http://brianwhitworth.com/STS/STS-
chapter1.pdf Note that the full design is not
yet specified - we intend to develop this working with
collaborators in a feedback process, so if you want to be
part of that let us know. I am working with Rob Friedman and
Michael Browstein on this project, so I copy this
email to them also. all the best
Brian Whitworth
Cheers
Mathieu
----- Original Message -----
From: Stefan Merten <smerten oekonux.de>
Date: Monday, September 7, 2009 10:50 pm
Subject: Re: [jox] Re: Peer Review
To: journal oekonux.org
Cc: Stefan Merten <smerten oekonux.de>
Dear Mathieu, CSPP Board Members, all!
Last week (9 days ago) Mathieu O'Neil wrote:
OK, I contacted one of the authors of the FM paper and he kindly
sent me links to his second part as well as another relevant
chapter.
Mathieu sent the links privately. Indeed a very useful paper. I'll
respond to this here because otherwise the communication
becomes to
confusing for me ;-) .
In his mail he also mentioned that they are seeking feedback from
people on these issues. I responded that would be possible
but that
I was acting as part of a group and queried whether it would
be OK
to circulate the papers to this group (without really saying
who was
in it though I had originally sent a link to this list's
archive).
They are welcome. But then we need the right to put the link here
(it's a public link anyway so this should not matter too much).
@Mathieu: I hope you won't mind that I quote the following questions
from your private mail here.
Last week (7 days ago) Mathieu O'Neil wrote:
I guess the main questions are: which features would we want?
In general I think those guys are very much on the same track
as we
:-) . I think the key question to solve I already mentioned in
my post
from `Mon, 24 Aug 2009 12:25:45 +0200`__. To prevent leaking
too much
information from the paper here I only reply to what is new or very
different from my suggestions there or comments from others
made here
already. I relate to the chapter "DEMOCRATIC ONLINE KNOWLEDGE
EXCHANGE".
__ http://www.oekonux.org/journal/list/archive/msg00091.html
* Multi-level ratings
Similar to my previous suggestion the submissions
can and
should be
given a number of levels reflecting how the
reviewers /
editorial board assesses the submission. That solves the
problem of the binary
accept / reject scheme. On a website it is also
easy to flag
articles appropriately.
Also using several rating dimensions is a good
idea. What
is needed,
however, is to reduce the many dimensions to a
single
number to give
orientation to readers. May be the scientific vs.
activist category
could be one such dimension.
I think the reviewers should still have the
responsibility to rate
an article. May be the reader rating could be one
more
dimension -
given that there are enough ratings from the readers.
Readers need to have an option to tune what
dimensions
they consider
most important. Technically this would mean to
have different
reports using a reader's weights. That is
technically simple.
* Submissions
I think every submission should be addressed to
the
editorial board.
IMHO this also adds a level of responsibility on
the part
of the
submitters: It's a difference whether you just
drop
something in an
electronic system or ask real people to consider
your
submission.
This doesn't mean that a submission is withheld
from the
website. It
can be put there immediately if the author wishes so.
* Anonymous contributions
In general I don't like anonymous contributions as
well
as I don't
like near-anonymous contributions from obvious
pseudonyms. I see
that creation of an account actually feels like an
obstacle - at
least to me. But nonetheless I don't like
anonymous
contributions. IMHO this at least reduces the spam
problem
very much.
Which of those
we want can we have without too much hassle?
I think most if not all of their suggestions and our ideas are
technically feasible. IMHO the real work (i.e. human labor) is
reading> submissions, thinking about them and making decisions.
This real work
needs to be done anyway. The rest can and should be left to the
machine.
Grüße
Stefan
______________________________
http://www.oekonux.org/journal
****
Dr Mathieu O'Neil
Adjunct Research Fellow
Australian Demographic and Social Research Institute
College of Arts and Social Science
The Australian National University
E-mail: mathieu.oneil anu.edu.au
Tel.: (61 02) 61 25 38 00
Web: http://adsri.anu.edu.au/people/visitors/mathieu.php
Mail: Coombs Building, 9
Canberra, ACT 0200 - AUSTRALIA
[2 text/html]
______________________________
http://www.oekonux.org/journal