Message 00120 [Homepage] [Navigation]
Thread: joxT00000 Message: 82/176 L25 [In date index] [In thread index]
[First in Thread] [Last in Thread] [Date Next] [Date Prev]
[Next in Thread] [Prev in Thread] [Next Thread] [Prev Thread]

Re : Re: [jox] Re: Peer Review

[Converted from multipart/alternative]

[1 text/plain]
Hi all

Thanks for comments, I was indeed beginning to wonder about the radio silence!...
Alles gut!
What I see ahead:
1-I will try to summarize our discussions re peer review to create a final-ish version of the journal description. This can be used to send to potential reviewers as well as to populate a CSPP webpage on the ox site.
2-I will draft a cover letter to send to potential reviewers and post it here.


----- Message d'origine -----
De: Athina Karatzogianni <athina.k>
Date: Mardi, 20 Octobre 2009, 10:32 am
Objet: Re: [jox] Re: Peer Review
À: journal
Cc: Stefan Merten <smerten>

[Converted from multipart/alternative]

[1 text/plain]
Hi Everyone,

I just read Stefan and I think I would agree fully with his 
ideas on the
project. Just a note if it is helpful, there are journals 
printed on demand,
the reason I know is because of my contribution to this journal 
which is has
a print on demand service, so it can be done, please have a look 
here for
the example I am talking about:

A second point, I do agree that the alias idea is not good and 
the light and
heavy mode of reviewing in my opinion is unfair to the authors 
reviewed as a
process. Everyone should go under similar standards of scrutiny 
for reasons
of fairness.


On Tue, Oct 20, 2009 at 7:05 AM, Stefan Merten 
<smerten> wrote:

Hi Mathieu and all!

In general I think we have reached a stage of rough consensus. The
recent silence on the list to me is a clear indicator for 
this. I'd
find it useful if the results of our discussion could be summarized.

My next most important task is clarifying a final but 
important detail
in the website setup - see [pox] - and then I think we can 
start a

I'm giving my opinion on the questions raised by Mathieu. 
Sorry for
any duplication.

3 weeks (24 days) ago Mathieu O'Neil wrote:
First I wanted to make a suggestion to address the concern 
raised by
StefanMn in relation to new reviewers changing the course of 
the project:
there could be two distinct groups.
One group, "reviewers" (not yet involved in the project), 
would be mostly
involved in peer reviewing.
Another group,"editors" (already involved), would be 
involved in peer
reviewing as well as overall project governance. Over time if 
reviewers take
part in discussions etc and are accepted they could become 
part of the
editorial team? Just a thought.

This matches my idea quite closely. My experience in project 
setups is
exactly this: Check how people perform in the "normal" 
community and
if you like it ask them whether they would like to take more

OK, in terms of the peer review process and the articles by 
Whitworth and
Friedman (WF) which I referenced a while ago - message below 
if you want to
have a look.
I think these articles offer some very useful ideas. But not 
all ideas
need to be put into practice. In any case they all require some
technological expertise...

I just started to get acquainted with the non-technical 
details of
Plone_ - which will be the technical basis of the website. So 
I'm not
yet an expert for using every feature of Plone but I know 
Plone is a
*very* rich "eco-system" with lots and lots of features. From 
this I
can draw some conclusions on what is possible probably.

.. _Plone:

Following are some thoughts and questions.

1_General principles
First here is a summary of what they propose in their new 
system, as
opposed to the traditional system

a) higher rating discrimination (a many-point scale, not just

One key advantage of the web: You have no space limitation and thus
can be more voluminous easily.

b) more submissions to be rated (rate all)

A question of available labor force basically.

c) more people to rate (community involvement)

Another key advantage of the web: Feedback channels.

d) different ways of rating (formal review vs. informal use 
Another advantage of the web where it is quite easy to measure use

So what we could have is a more transparent process where 
all submitted
articles are peer reviewed and then rated from [PHONE NUMBER REMOVED] (excellent) 
to -1 (don't
However in the WF system if authors decide they want to 
publish publicly
a - 1 article then it is published anyway. This is of course 
I think the editorial board should be the final decision maker here.
If an article is just rubbish it doesn't need to be published 
in our
journal - regardless on how much the author wants it.

In the web people are flooded by shit anyway and I think the 
people we
want to address with the journal are grateful if someone acts 
as a
filter for them.

WF suggest that + 5 articles could be published as print or 
otherwise> "frozen". Graham did mention print-on-demand which 
would be something to
think about for the future. Does anyone know anything about 
this? I don't..

Print-on-demand is a model for books. I don't know whether 
this works
for articles. ChristianS offers his book by a print-on-demand 
model so
he is an expert.

Note: If people want to keep commenting on articles (for 
readers) or
responding to comments (for authors) in my view that is fine 
but as was
discussed previously I don't think the idea of endlessly 
updating the body
of an article itself (as happens on Wikipedia) is appropriate 
for a journal.

Yes. May be we can take it this way: Improving an article (beyond
grammar and the like) is the sole responsibility of the original

3_Anonymous review choice
Allows authors to improve their paper privately. I think for 
research> papers they should all be reviewed in this way. Maybe 
we could have a choice
between "light" (one reviewer) and "heavy" (two-three 
reviewers) reviews
 but this may end up complicating the process a bit. So 
it might be better
to just have normal two reviews?

I think we should prevent making things too complicated. In practice
we will find some practice which works under the given 
constraints and
we can formalize this later.

4_Reader ratings
WF say these could be the done formally by votes or 
informally with mouse
clicks (number of views or downloads). Thoughts? Obviously 
informally needs
more technical monitoring.

So far I have no idea on what is possible in monitoring page views
with Plone. Sure we have an Apache in front of Plone which 
logs hits
but a useful monitoring needs some work - especially because the
caches in front of Plone may hide accesses from Plone. Since 
this is a
standard problem there might be already a solution to this 
problem out

5_Performance reports
WF propose that like students who can request grades from 
universities> contributors (such as authors) could request 
reports not just of
publications but of citations, number of comments, of 
downloads and views
generated. Reviewers could ask for records of their 
contributions to be sent
directly to institution.
Possible? Desirable?

For the possibility of numbers technically generated see above.

For the question of availability: If the numbers are available they
could be published right away. In any case I have no general 
problems> with such use since it creates / indicates more 
attention for our
journal and that is in the best interest of the journal.

6_View filters
This refers to only displaying content rated above "x" like 
on Slashdot I
Should it be done? Can it be done?

As I explained I think it could be multi-dimensional. It is 
certainly> doable but it evenly certainly needs some technical 
work (which I'm in
general ready to do). If we are lucky there is already a 
solution to
this problem out there.

7_Same again function
Automatically helps people find more of the same - ie find 
content rated
highly by those who rate like you.
Should it be done? Can it be done?

I'm not sure whether I understand this fully but I guess it is
technically doable.

WF suggest:
Logical flow
Others? Less?

See my suggestions in another post.

9_Progression of readers
I think most would agree that we should have reader input. 
If there is a
rating system generated by readers then they can offer a 
counterpoint to
'expert' and / or academic reviewers. WF suggest there could 
be a natural
path to associate reviewer, reviewer, associate editor, etc

Reader input / rating is fine.

This raises the question of who can be a reader? Do we require
credentials? Even more fundamentally, this raises the 
questions of identity
and privacy for readers and commenters.

The general reader should need no credentials in any case. Why limit
read access artificially?

For commenters a registration seems to be a useful thing.

BTW: In general it is easier to create a limitation right from the
start than introducing it later on.

WF say (and I agree) that we need contributors to register 
to avoid spam
and vandalism. They write that jounals should 'check the 
credentials of
_who_ is submitting not censor _what_ is submitted'. I think 
they mainly
mean authors though.
I also think they are talking about authors.

It is not entirely clear what they think about readers (a 
priori "expert"
reviewers are known in-house so not concerned by this issue). 
My preference
would be to encourage or demand real names for commenters and 
raters, on
principle. Many of the problems on Wikipedia (for example) 
derive from
anonymity and the attendant lack of responsibility which 
almost generates

Wikipedia is a nice example of how difficult it is to introduce
limitations later - though Wikipedia is not really comparable 
to a

Questions about this:
-- Can / should we have an automatic registration system for 
participants> or would people need to approach us to be 
registered and given access?

Plone has a sophisticated user and rights administration. As a
technical basis this certainly suffices.

For commenters there should be a registration which can be 
operated by
everyone. For anything else the editorial board needs to make a

-- Do we check that people who want to register are who they 
say they
are? If yes, how?

This check is practically impossible. If people want to hide their
identity they can. And that's fine with me. It would be good, 
however,> if they use a consistent alias in this case and don't 
act under more
than one alias.

For anything more than commenting we need to require people to give
names which are not obvious aliases / nick names - may be with
exceptions for artists. We are doing something serious here 
and if
people think they need to hide their identity they are in the wrong
place anyway.

There are other ideas such as having an automated  
system for logging and
tracking submissions but this could be set up later as there 
is already a
system for doing this - me :-) - whereas some of the points 
above have no
solution yet.

Plone has also a sophisticated system for supporting a publication
process. AFAICS this can be configured and I think it will be easy.

In short:
To everyone: what do we want?
To StefanMn and StefanMz: what can we have?

My cent.



Dr Athina Karatzogianni
Lecturer in Media, Culture and Society
The Dean's Representative (Chinese Partnerships)
Faculty of Arts and Social Sciences
The University of Hull
United Kingdom
T: ++44 (0) 1482 46 5790
F: ++44 (0) 1482 466107

Check out Athina's work

Check Virtual Communication Collaboration and Conflict (Virt3C) 

[2 text/html]

Dr Mathieu O'Neil
Adjunct Research Fellow
Australian Demographic and Social Research Institute
College of Arts and Social Science
The Australian National University

E-mail: mathieu.oneil
Tel.: (61 02) 61 25 38 00
Mail: Coombs Building, 9
Canberra, ACT 0200 - AUSTRALIA

[2 text/html]

Thread: joxT00000 Message: 82/176 L25 [In date index] [In thread index]
Message 00120 [Homepage] [Navigation]