Message 00215 [Homepage] [Navigation]
Thread: joxT00207 Message: 6/6 L2 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

[jox] Tagging as rating (was: Re: Concerns and suggestions)

A happy new year to all of you!

Thank you, StefanMn, explaining your position in more detail why a 
differentiated pre-selection is important for our journal. My concerns 
mainly came from the picture of being rated out in a binary rating due 
to weaknesses on single dimensions (like language or controversial 

Regarding your two suggestions I like the idea of tagging a submission 
very much. This feels like being quite near to peer trust models which 
clearly base on subjectivity, which is not their weak point, but their 
strength. What there yet has to be solved is the question of how tagging 
is translated to the paper being accepted or not.

So I would favor the tagging model, but I can live with the options 
model, too, if others mainly opt for it.


On 2009-12-21 13:20, Stefan Merten wrote:
In any case one of my suggestions also is an option model. However,
I'd suggest to have two modes the submitter can choose from:

* Binary rating

  A paper is submitted and it is accepted by the journal responsibles
  or not. Papers which are less-than-optimal are not accepted in this

* Differentiated rating

  A submitter can ask for a differentiated rating from the journal
  responsibles. Papers which are less-than-optimal can be accepted in
  this mode. There is a perspective where readers can see and may be
  even use the rating for their own perspectives.

  In case there are journal responsibles who for whatever reason
  refuse to turn their expert knowledge into a differentiated rating
 - that seems to be one point of Athina's concerns - they probably
 can not help with such submissions.

If we leave this choice to the submitters then we can also check what
they prefer and thus make an experiment. I'd also think the concerns
of Athina are void then because the submitter chooses herself on how
to be treated and I can see nobody who suffers in this case.

The second suggestion is to create several rubrics where all papers
are sorted into. For instance there could be rubrics for

* Controversial

* Theoretical

* Empirical

* Critical

* Affirmative

* Activist report

* Bad English

This would be similar to a differentiated rating but it reduces the
rating to a single dimension which makes it difficult for a critical
paper based on empirical studies...

Also I'd think it would be useful to have rubrics for the topics we
agreed on:

  Topics of interest include, but are not limited to: the political
  economy of peer production; peer production and expertise; critical
  theory and peer production; peer production and exchange; peer
  production and social movements; peer production as an alternative
  to capitalism; peer production and capitalist cooptation;
 governance in peer projects; peer production and ethics; the peer
 production of hardware; peer production and feminism; peer
 production, industry and ecology.


When I think longer about it all this really calls for tags which are
attached to a submission...

The problem with this approach is that the responsibles need to agree
on a set of tags.

Well, the second suggestion is probably a bad compromise. The first
suggestion is very close to what you, Mathieu, suggested already and
from my POV even addresses some of Athina's concerns better.

What we still need to think about is creating rubrics and/or tags to
sort submissions according to topics.

Start here:

Thread: joxT00207 Message: 6/6 L2 [In date index] [In thread index]
Message 00215 [Homepage] [Navigation]