Message 00085 [Homepage] [Navigation]
Thread: joxT00000 Message: 69/176 L16 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

[jox] Re: Peer Review [Was: RE: Review process



Following on from Mathieu and Athina's comments:

This is all getting rather binary: high prestige model non-transparent
blind reviews versus transparency and potential flame wars and spam.

We aren't the first to be creating a kind of parallel universe to the
academic journals; it's something the old left have done for years, the
newer left are doing now, and we're not the only ones concentrating on
the commons one way or another. Clearly the old left approach is not
relevant (review consists of deciding if the party line is being
followed). So I thought I'd have a look at some of the newer ones. It's
suprisingly hard to see what the process is:

commoner.org.uk (Massimo de Angelis and friends) - can't find how to
submit an article.

multitudes.samizdat.net
http://multitudes.samizdat.net/Submission-and-evaluation
process modelled on academic one

freesoftwaremagazine.com (more of a magazine than a journal)
Individual editor reviews http://www.freesoftwaremagazine.com/write_for_us

www.metamute.org/content/magazine (IMO very mixed quality of articles)
can't find review process

www.posseweb.net (negri and friends)
can't find how to submit an article

http://turbulence.org.uk
Current CFP: http://turbulence.org.uk/2009/06/turbulence-call-for-art/
Can't find review process

[Nearly all these have a print version; given how easy it is to do POD I
think we should consider this fairly early on (maybe talk to openmute
www.openmute.org and using their pod setup?)]


So the review processes seem to split into two: academic style, and
informal non-transparent 'an editor does it'. I haven't found anyone
deliberately doing a transparent version.

So how about we be the ones to do it, but as an experiment? We offer
blind review as the default route, but authors willing to be part of the
experiment submit their article for open commenting and feedback
instead. The final version of each article is flagged as to which review
process was used, and after a period (three issues?) we wrap up with
some conclusions on the merits of each system and assessment of the
quality they generate.

The big downside to this is it's likely to involve double the work for
us :-(

Graham
PS. Another thought: anyone have experience of spip
(http://www.spip.net) ? It was originally created to manage web journals..




Mathieu O'Neil wrote:

2- For this we need good reviewers and good comments. There could be a
system where open discussion on a list leads to editorial comments being
appended to papers. Not sure. I don’t think we should rely on "anyone can
comment" to do this job - no-one may comment or comments may be mediocre.

3- There needs to be some clear guidelins for an open comment process:
-- closed editorial list / closed registration process?
-- deadlines for comments to be made?

4- It is clear that different review processes could be useful. We need to
define precisely the different review processes: blind or not, open or not,
etc.

5- The second part of the paper cited above may have some interesting
leads...

6- In conclusion: we eventually need to get some more people on the
editorial board to help advance how the review process works. We will need
some input from the people we will be approaching to work with us. So we
need to progress the rest of the "charter" so we can start approaching
people.

Cheers,
Mathieu


-----Original Message-----
From: owner-journal oekonux.org [owner-journal oekonux.org] On Behalf
Of Stefan Merten
Sent: Monday, August 17, 2009 6:42 PM
To: journal oekonux.org
Cc: Stefan Merten
Subject: Review process (was: [jox] New Draft CFP)

Hi Mathieu, Athina, all!

Last week (11 days ago) Mathieu O'Neil wrote:
d) Regarding peer review

I suggested the following: for research papers, authors can request a
traditional double blind review. But following this process, research papers
(as all other submissions) will be collectively discussed on the list.

I think it is an interesting idea to have different processes.
However, I'm not sure about the consequences. What do others think?

For those not too deep into the traditional process: Could you please
explain what are the features of the traditional double blind review?

Last week (10 days ago) Athina Karatzogianni wrote:
About d, I think it would be prudent to think about the implications of
discussing papers openly on a list. perhaps people will be much less
critical of a work once it is openly discussed.

That was a concern mentioned before. If this point is important then
it would indeed impact the quality of the journal. This would be bad.

Who would be able to see
this discussion?

It depends. Oekonux lists are usually published on the site but we can
also have a non-archived list. On such a mailing list a discussion
would be open among the editorial board but closed to the public.

Also there can be exchange based on personal e-mail. However, I'd find
it bad for transparency if regular personal e-mail exchange would
occur unless it is between persons who are working closely together on
a particular task - such as reviewing a contribution. To prevent this
I'd rather suggest a second, non-archived mailing list.

what if one of us wanted to publish a paper, would we look
at the reviewers comments while they were formulating them?

What's wrong with this?

I think some
thought should be paid there. The tradition is to have 2-3 blind reviewers
for a paper.

See above. Can you please explain what "blind" means exactly?

I dont see and please explain to me how when ten people have a
long discussion over an email list, quality and speed improve. I think it
will be quite the opposite.

IMHO this depends much on the culture of such a list. I know most of
the persons on this list personally and most for quite some time now
and I don't think that there will be unnecessary discussion.

Anyway I understood that there will be explicitly assigned reviewers
for each contribution - 2-3 sounds good to me. They are responsible to
review the particular contribution and alone for reasons of lack of
time people will probably trust the judgement of the reviewers.

Blind reviewing most of the time works in favor
of the author. Discussing between us endlessly a paper [unless it is
highly
controversial and only after it has been blindly reviewed] I think will be
a
waste of time and effort.

Endless is discussion is not very probable IMHO. If a contribution is
too controversial it simply will not be included. That would at least
mean an orientation in consensus in the editorial board (where
consensus means that nobody *has to* object).


						Grüße

						Stefan
______________________________
http://www.oekonux.org/journal

______________________________
http://www.oekonux.org/journal

______________________________
http://www.oekonux.org/journal



Thread: joxT00000 Message: 69/176 L16 [In date index] [In thread index]
Message 00085 [Homepage] [Navigation]