Message 00209 [Homepage] [Navigation]
Thread: joxT00207 Message: 3/6 L2 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re: [jox] Multi-rating mode of evaluation

Hi Biella

The email was sent on December 6.

I'll reprint it below, adding in my suggestion for the possibility of electing to be rated according to english and controversiality.



Hi all

OK, well there is obviously some serious opposition to the expert
rating system (sorry Brian!). At the same time no-one seems to have a
problem with reader ratings... ?

Anyway, I have tried to come up with a new proposal based on the paper
by Tony Prug posted by Johann. I have modified his system a bit for
two main reasons: first, in order to tighten and quicken the process.
Second, to find a balance between openness and the risk of outsiders
inappropriately using material before it has been officially released
or rejected.


Prospective authors submit a proposal to the list.

All list members vet this proposal during a reasonable period of time
(1-2 weeks?): is it appropriate for the journal, are there missing
arguments or references?

Authors write their submission.

Authors submit to the journal, specifying (if applicable) whether they want to be rated according to English / controversiality. 

The submission is posted by the editor to a password-protected part of the website for reference who also alerts the list that he has done so.

The editor suggests two expert reviewers (submissions welcome).

The two expert reviewers read and evaluate the submission during a
reasonable period of time (3-4 weeks?). Reviewers can coordinate their reviews. Reviewers decide whether papers should be rated according to English / controversiality. 

Reviewers post their reviews and recommendations to a password-
protected part of site and alert the list that they have done so. 

The list discusses this during a reasonable period of time (1-
2 weeks?).

During this time consensus emerges: publish, revise and resubmit (to
two other reviewers, for example?) or reject, or

During this time consensus does not emerge: the decision then moves to
a formal vote on the Governance Board: publish, revise and resubmit
(to two other reviewers, for example) or reject.

Submission published.

Readers can comment [and rate?].

Authors can respond in comments section [and add links in the text to
relevant comments and responses? No updating of text though].


----- Original Message -----
From: Gabriella Coleman <biella>
Date: Tuesday, December 15, 2009 1:17 am
Subject: Re: [jox] Multi-rating mode of evaluation
To: journal

Hi Mathieu,

I seemed to miss the proposal for proposing new SC members. Can 
you point me to the email again?



Mathieu O'Neil wrote:
[Converted from text/html]

HI Athina, Stephan, all

I dont think just because one person has their mind set on something
constitutes a sufficient reason to do it, whoever that person 
may be.
I think its important to keep in mind what people want. A 
number of
people have expressed serious reservations about rating
indiscriminately. At the same time StefanMn has argued for ratings:
but why? If I remember correctly it was mainly to allow papers that
were either a) "controversial" (ie political, radical) or b) written
by non-English speakers, to be published.

My suggestion is to compromise: normally published papers are not
rated. They are produced along the lines I suggested in my post 
called>"Alternative peer review system", inspired by Toni Prug's 
piece. But
when authors or reviewers feel that the paper are concerned by either
of the categories mentioned above, then authors can ask for and
reviewers can suggest that they are rathed in these categories. That
way papers which would either not be published or take endless
negotiation or fixing up to be published can be.

I agree with Athina that the debate needs to be wrapped up otherwise
energy will continue to drain out of the project.

The same goes with how to define who can be part of the SC. I have
made a proposal that seemed to me fair and clear a number of times,
I'm not going to repeat it again. Well some new people have been
suggested but I didnt want to contact anyone until there was



On 12/12/09, Athina Karatzogianni <athina.k> wrote:
     [Converted from multipart/alternative]

     [1 text/plain]
     Hi Stefan

     I think you have your mind set on 
this, and so you should
     just go ahead and
     experiment with your rating system and 
she where it gets us.
     You want to try
     this out, so just do it!!

     Just to say though that your comment: 
'Yes. And this is more
     like classical
     democracy then. But the readers can 
only give their opinion
     on pre-selected
     items' to me is very problematic, from 
the "classical
     democracy" perspective
     you are referring to. So yes, I dont 
mean to be difficult,
     but I do not
     think continuing this debate will have 
any more fruition and
     is just
     delaying the project. At the same time 
we are getting into a
     debate that is
     better left inside the journal's 
future content pages and
     not theoretically
     debated in pre-organizational terms 
and times

     I need some coffee desperately


     On Fri, Dec 11, 2009 at 7:28 PM, 
Stefan Merten
     <smerten> wrote:

     > Hi Brian and all!
     > I'd like to reply to this from the 
perspective of peer
     > Last week (7 days ago) bwhitworth wrote:
     > > Statements like "We should publish 
only papers that we
     agree are fit
     > > for publication" or "We should 
..." in general assume
     that we
     > > control the journal. Our paper at
     > >
     >>     article/view/2609/2248
     > > opposes that control mentality to 
introduce the ideal of
     > > in academic publishing, i.e. 
government by the people
     for the people
     > Well, I'd say yes for the openness 
and transparency but no
     for those
     > who make the choice - at least for a 
journal like this.
     > Peer production projects are not 
democratic but have
     maintainers (aka
     > leaders) who are listened to by 
volunteers. The
     maintainers are
     > maintainers not by any alienated 
facility but because
     their work is
     > useful for the project at hand.
     > This is how I see it for a journal 
like this as well.
     People *do*
     > subscribe to such a journal 
*because* they trust the
     > persons to make a good choice for 
them because they don't
     have the
     > time / knowledge / ... for this 
work. Just like people
     *do* choose
     > Ubuntu *because* they trust the 
Ubuntu maintainers to do a
     good job.
     > > Likewise the ratings of registered 
readers, while
     informal, are not
     > > unexpected nor imposed. The public 
is always entitled to
     > > opinion. The system need only 
identify and ban spammes
     and trolls,
     > > as Wikipedia does. The view of the 
public should not be
     a secret, so
     > > people can rate what they read.
     > Yes. And this is more like classical 
democracy then. But
     the readers
     > can only give their opinion on pre-
selected items. As I
     argued the
     > pre-selection is exactly the task of 
the persons
     responsible for the
     > journal.
     > ______________________________

     Dr Athina Karatzogianni
     Lecturer in Media, Culture and Society
     The Dean's Representative (Chinese 
Partnerships)>     Faculty of Arts and 
Social Sciences
     The University of Hull
     United Kingdom
     HU6 7RX
     T: ++44 (0) 1482 46 5790
     F: ++44 (0) 1482 466107

     Check out Athina's work

     Check Virtual Communication 
Collaboration and Conflict
     (Virt3C) Conference

     [2 text/html]

Dr Mathieu O'Neil
Adjunct Research Fellow
Australian Demographic and Social Research Institute
College of Arts and Social Science
The Australian National University
email: mathieu.oneil



Gabriella Coleman, Assistant Professor
Department of Media, Culture, & Communication
New York University
239 Greene St, 7th floor
NY NY 10003

****<br />Dr Mathieu O'Neil<br />Adjunct Research Fellow<br />Australian Demographic and Social Research Institute<br />College of Arts and Social Science<br />The Australian National University <br />email: mathieu.oneil<br />web:<br /><br />


Thread: joxT00207 Message: 3/6 L2 [In date index] [In thread index]
Message 00209 [Homepage] [Navigation]