Message 00211 [Homepage] [Navigation]
Thread: joxT00189 Message: 8/77 L3 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Concerns - conservative vs. progressive (was: Re: Re: [jox] Multi-rating mode of evaluation)

Hi all!

First I should probably say that I'm sorry. I had hoped that I was
able to follow Mathieu's wish to stop this debate but at least to me
it is too important to stop it here. I also tried to just let pass by
some time hoping that time will make it irrelevant for me.
Unfortunately the desired effect didn't take place :-/ . So at the
very least I think I need to post my concerns.

On the other hand from my perspective the aspects *underlying* this
debate are important for this whole journal project. I'd like to
remind people of the (first part of the) mission statement:

  Critical Studies in Peer Production (CSPP) seeks high-quality
  contributions from researchers and practitioners of peer production.
  We understand peer production as a mode of commons-based and
  oriented production in which participation is voluntary and
  predicated on the self-selection of tasks. Notable examples are the
  collaborative development of Free Software projects and of the
  Wikipedia online encyclopedia. Through the analysis of the forms,
  operations, and contradictions of peer producing communities in
  contemporary capitalist society, the journal aims to open up new
  perspectives on the implications of peer production for social


The mission statement clearly talks about peer production and gives a
short outline making clear what we understand by peer production.

In this mail I'll try to outline what I think is wrong with some
positions recently posted on this list - so please do not expect this
mail to be very constructive. In another post I'll write during the
next few days there will be also some constructive ideas on how to
deal with the situation.

I'll quote Athina's mail but some explicitly and others in spirit
share this position.

2 weeks (14 days) ago Athina Karatzogianni wrote:
My view is indeed that rating is unnecessary for many reasons. These are my
more important ones but I can think of many more:

   - you are treating the readers as idiots who cant do it themselves

This is in fact the only aspect which mentions the readers. And the
message of this statement basically is: Why the hell should we help
the readers? They are smart enough themselves.

Well, I find this position - hmm - interesting. It is no question that
readers are smart. But again: If readers are smart enough to rate then
they don't need a journal at all. They can just use Google. These
readers don't need a journal which pre-selects texts for them. So
readers who don't need a rating and therefore they don't need a
journal and thus they are not the target group of the journal.

However, peer production is about creating use value. And I think such
a journal *does* produce use value for some people on this planet. If
I wouldn't think so I would not invest a single minute in this
project. After all it's not me who needs such a journal... So for the
target group of the journal the journal can only be interesting
*because* it pre-selects.

As a logical extension to a plain binary pre-selection - in or out - I
think the use value for readers is bigger when they get more
information on what could be interesting for particularly them. That
is what a multi-dimensional rating system is meant for.

   - you are essentially playing favorites with the texts

Well, that is exactly the nature of a journal. A journal is a
selection mechanism to select texts which the persons responsible for
the journal find interesting, matching the mission of the journal and
being of good quality. That is of course playing favorites. But that
is exactly what a journal (or any other information filter) does.

And while I'm at it: A journal with an anything-goes attitude is of
course completely useless because it doesn't filter. Such an approach
would end up in the tyranny of structurelessness.

So to keep a journal useful it needs to be very selective. That is
basically the point Brian makes in his criticism in classical
scientific journals. The problem is that without a mechanism like
Brian proposes it is impossible to be useful *and* present
cutting/bleeding-edge stuff. Unfortunately I heard not one suggestion
to *this* problem from Athina and others sharing her position. I only
can hope that my worry that they favor an anything-goes attitude is
simply wrong.

Also in peer production there is always "playing favorites". For
example in Free Software the code that works best ends up in the big
distributions - often after being improved in a peer production
process like Toni suggested. Other code is simply discarded. And users
of peer production products agree with this selection - they even
choose a product for exactly this selection.

   - rating is a very subjective business and disagreeing with someone's
   position irrespective of the quality of the paper might confuse the two, if
   you disagree but the work is great how do you rate? The mechanism you
   describe really doesnt say anything to me

Well, this statement assumes that there is some super-subjective,
abstract truth: "but the work is great". There isn't. God is dead.

Peer production accepts this. In peer production nobody hunts for some
abstract truth. And if some peer producers do at least they start
working on concrete results instead of waiting for some divine help.
What they also do is to make things transparent and react on feedback.

To me everyone who thinks an endeavor like this could produce
something which is not subjective is heavily mislead. Each decision is
subjective. Employing the principles of peer production I find it
important to make things as transparent as possible and be open for
feedback. A multi-dimensional rating system could be a quick indicator
of that transparency.

The minor point made by the statement about disagreeing with someone's
position also clearly would be solved by a multi-dimensional rating
system. You could express that you don't like the position - or
rather: it is not clearly part of the mission - but nonetheless it is
a great paper. If you don't have the option to express this such a
paper *needs* to be rejected! Exactly the problem Brian points out.

   - especially when on cutting edge research matters, it is difficult to
   even rate altogether

Well, the only solution without a multi-dimensional rating system is
to reject the paper. That is of course much easier but I thought we
are trying to prevent this.

And also: Just because something might be difficult this alone is not
a reason not to try it. If Richard Stallman or Jimbo Wales would have
said "But it's difficult to do this" then we would have neither Free
Software nor Wikipedia.

What would useful is to make it transparent on how the rating comes
about. But everyone with a mind in her head *knows* that this whole
thing is subjective.

   - you are creating a HIERARCHY of  papers and  i do find hierarchies in
   general quite distasteful. For a progressive journal on p2p this could be
   very problematic.

Basically the same again: Inside or outside a journal also creates a
hierarchy. The only point is that this rating is binary and
necessarily cuts out interesting stuff.

Also in peer production hierarchies are usually not considered a
problem. Again they are helpful for the users and users *trust* those
who hierarchies.

So for me this position once was progressive. It was progressive in
capitalism when equality was a top goal.

In peer production people return to the more basic values and they
come to other conclusions. Simply being against hierarchies today
therefore I'd consider conservative. If you agree that peer production
makes sense then it is more progressive to watch what really happens
in peer production and why people *voluntarily* make themselves
subject to such conditions - although political correctness can not
understand this.

   - What is a better paper and who decides? Majority decision making does
   not work well in real life, why do you want to transfer it onto an
   experimental progressive journal?

You need to make a decision anyway. If you are not ready to take the
responsibility to make a decision then you are probably in the wrong
movie. The best you can do is to make the decision as transparent as
possible. Then people have a basis on which they can build up trust in
your decisions. That is how peer production works.

   - Also, people on it might feel uncomfortable with the rating and it is a
   very technocratic thing to do altogether.

The only ones who I can think of who might be uncomfortable with are
the authors. If there are really readers who do not want to see any
rating it would be easy to create a view without any rating. Just a
plain unordered list of articles then.

For the authors, however, there could be an option that the whole
article is not published if they are not happy with the results of the
submission process. In this case I'd agree that this is kept secret
until the submission process is completed. Of course we need to make
it clear that there will be a rating being published with the article
so authors know in advance.

Also I could imagine that a rating could even be helpful for authors.
For instance if there is one rating category about the amount of
empirical data and there is a lack of it for some article someone else
could notice this and help out with empirical data which in effect
will improve the article in a next step. That is how peer production
works and as an author I'd consider this a chance and not an

Also a peer producer can not simply expect that her contribution is
accepted without any further consideration. There *is* further
consideration and only if the contribution is (made) useful it is

   I mean this is not a marketing
   product and we dont sell DVD players!!

Again this is a position which totally neglects the user's view. If I
buy a non-trivial product I spend some time checking out what other's
say about it. Since the Internet is common this is fortunately easy.
But I'd really prefer a single source which tells me the truth which
is relevant for me.

If I have some spare time and would like to read something then I'm
happy is someone who I trust helps me to make a choice. In this sense
a journal has a very similar role as a market overview which compares
different products and rates them. Also the better ones of those
market overviews do multi-dimensional ratings so the reader can choose
what dimension is most important for her. In fact your idea is a very
good example on how we *should* operate.

   - Lastly personally trust very few researchers' opinion about research to
   begin with. Do you trust everyone? Do you trust the majority?

This refers to trust. Indeed trust is a central concept for peer
production. Users of peer production products *do* trust the
producers. Now this position just cited basically says: Don't trust
anybody. Of course this is impossible because you need someone who
helps you and you need to trust others.

But peer production users are not simply stupid. They really *can*
trust peer producers because peer producers have no alienated reason
to produce. They just produce to create use value and though they can
do this better or worse and though their result might match your
expectations or not there is basically no reason to not trust them.
I'd really hope that this journal works on this premises.

Nevertheless, I told Mathieu that all you can  do is experiment see how it
goes in my personal communication with him.

Well, that is something I really resent. Behind the scenes
communication is not only bad for peer production projects but also
for those who follow political correctness. Indeed this alerts me and
I only can hope that Mathieu as the maintainer of this project tries
hard to send any such attempt for private communication back to this

In which case, please go ahead
and I just register my concerns on the issue. Only trial and error can
settle this one.

I'd really hope that we can find some solution which is not black or
white. And I'm also for trying things out. But then there must be an
option for trying things out. But more about this in my next mail.

But back for the more general points. Brian named this position
political correctness and I think he is very right. In the past Athina
more than once came to the conclusion that peer production doesn't
care about peer production. For instance see her contribution_ to the
last Oekonux conference. That is simply true. If peer production
would have been political correct Linus Torvalds and Andrew Tanenbaum
would probably still discuss since 18 years.

.. _contribution:

The emancipatory potential of peer production is *beyond* political
correctness - though this might be hard to grasp for classical
leftists. And I know what I'm talking about because a few facets of
this truth I had to learn the hard way myself. In fact since Oekonux I
think political correctness is a conservative concept. It is of course
an important tool for capitalism. But this journal is about peer
production and in this realm other things are progressive. For me this
is indeed an important research topic.

The mission statement expresses very well why I think this journal is
important. What I'm trying to do is to be true to the mission. Part of
this is to think in a readers-first way and that is why I think
readers needs need to come first. Another part of this IMHO is to work
in a fashion at least compatible with peer production. Political
correctness thus has to be checked whether it matches principles in
peer production. If not I think we should *not* employ it but instead
employ what we see in peer production.

Phew - now I feel at least a bit better ;-) . As promised I'll write a
more constructive mail soon (*real* soon ;-) because I have some days
of vacation...).



Thread: joxT00189 Message: 8/77 L3 [In date index] [In thread index]
Message 00211 [Homepage] [Navigation]