A Conversation for Design for a New Peer Review System

more thoughts

Post 1

Martin Harper

Oh - a bunch of stuff... smiley - smiley

Review Fora:
-----------

Collaborative Workshop - a good way to publicise this would be for the editors to use it for 'this weeks topic', at least initially till it gets itself going. The danger for all the review forums is that they may turn into graveyards (like the writing workshop now), so that kind of official involvement has to be a good thing - and it'd take no more time than it does now... smiley - biggrin

Alternative Workshop - I hope to see a new section of the guide for humour/fiction/poetry created sooner or later, anyway - this has been mentioned as a blue sky aim for some time - I hope it gets here quickly - then we can rename this 'Alternative Peer Review' and use it as just that. smiley - smiley

Life PR / Universe PR / Everything PR - people might get confused about which category they should submit to - perhaps keep a general PR forum, and a scout (or two) can move them into the specific ones as they read them - which would also be a good way to make sure that every entry gets a look by a Scout - and give people a specific place to go for 'fresh meat'. But that's something for the future, of course. smiley - smiley

In Blue Skies, I'm looking forward to the Update Peer Review... In the mean time, maybe an Update Workshop would be a useful plan - that's the one type of entry that under the proposed scheme would need to be moved to the workshop.

Entry data box:
--------------

I'd like to see options for the author to flag the entry as many things. Random ideas as follows:

Work in Progress: I'm working on this, and I hope it'll be finished soon. This flag times out after a while.
Not for Review: I don't want people picking holes in this, for my own reasons. Or it's already in some kind of unofficial directory, which is enough for me.
Personal/Place/Party: This is a page about me, my life, a virtual pub, 'foxy manor' place, a party, some kind of game, etc.
Abandoned: I have no further interest in this whatsoever - do with it as you will.
Real: This is an entry which I feel is suitable for the edited guide, but I haven't submitted it to PR. Perhaps I'm waiting for a Scout to agree with me, or I'm not that confident, or whatever. {needs a different name, though - 'straight edge'? Or just use default for this?}
Alternative: This is a piece of humour/fiction/poetry - see 'Real'
Collaborative: I'm working on this with a bunch of other people - and I'd love you to join in!
Update: This is an update/new info to another entry. Includes a link to the entry being updated. The entry being updated automatically gets a link to this one in either 'referenced entries', or 'updates'.
Default: I haven't ticked a box yet.

In addition, stuff which is settable by the italics - once set these things are permanent and replace 'default' or whatever.

Pending, Edited: as per now. Add links to the original entry.
Sub-Edited: entry has been sub-edited. Add link to original.
Recommended, Help Page: as now
In Guide: {needs new name} - there is a pending/edited/sub-edited version of this entry. Add a link to it.

For entries which have been through the system, there should be a 'edited version available' flag (rather than 'real', or 'Peer Review: _link_', or w) with a link to the version which has been sub-edited. Similarly, there should be a 'see the original' sticker on the edited entry back to the unedited version.

All these things should appear in the search results, and it should be possible to specify that you are only searching for, for example, 'alternative' entries.

Submission:
----------

The author should always be able to submit their entry, obviously. If they abuse that, then they'll probably get warned off by a Scout or two first, which will probably be enough to solve the problem. I can't imagine ever needing to involve the italics - but in a pinch, they could sort it out. smiley - shrug

Scouts should be able to submit abandoned and default entries wherever they like. Real, Alternative, Collaborative entries should be submittable to the appropriate places. Everything else is unsubmittable. And, naturally, the author should first be asked if they mind, and given a one week grace period, and yada yada yada. Except for abandoned entries, of course. smiley - winkeye

I'm not sure about non-Scouts. Obviously they should be able to do whatever they like with abandoned entries, but what about the rest? smiley - erm

Moving and Restarting:
---------------------

I think authors should be able to move things as well as restart - let them make the decision over whether they should restart for clarity. In general, I'd not want to see to much restarting.

I'd also like the ability for authors to temporarily suspend their entries from review forums - hiding them for a while. The 'Colorado' entry is a case in point of how this would help. Suspended entries would be unpickable.

If Scouts move something, then I'd like the propose move, second move system to be automated - and possibly require a thirding. If a move is proposed to two different places, or if a Scout clicks a button to say they disagree with a proposed move, then an email to the scouts egroup should be generated, and after discussion the italics can fix it. Proposals should probably be hidden from non-Scouts, but Scouts should probably post appropriately anyway (eg, I feel this entry is too much like fiction to get into the guide, yada, yada, yada).

Review Forums:
-------------

I think whether an entry has been posted to by a Scout should only be viewable by Scouts - we don't want to turn this into Scout Review... smiley - smiley

A link to the previous submission in case of restarts might be a good idea too. And, since these things will move, a mention of what forum it was submitted to first would be a neat idea. If we ever get Update PR, then an automatic link to the entry being updated should be added. In the case of Scouts submitting on the authors behalf, an automatic link to the author's space should be added.


more thoughts

Post 2

Mark Moxon

Review Fora - yep, I like the Update idea too, so I'll add it in as something to consider.

Entry Data box - good concept, way too complex, IMHO. As far as Peer Review is concerned, we only really care about whether something is reviewable or not, and that's the 'Not for Review' flag. Not sure that further statuses are relevant to Review Forums, but they could have uses beyond it...

Linking between Edited Entries and the original ones, though, is something we've always wanted to do (we have an internal tool that shows that link already). We do need to implement a proper superseding system, and it's on the list... but it's not really relevant to the proposal here either. Nice concept, though. smiley - smiley

Submission - given the 'Not for Review' option, I think anyone should be able to submit entries. If people don't want their entries to be submittable, they should tick the 'Not for Review' box - simple!

Moving and Restarting - I rather like the idea of forcing a restart when moving reviews around. It forces a new mission statement to be created, and given the different functions of the different Review Forums, that's a good thing, IMHO.

Implementing a system to formalise the seconding system for Scouts moving things might be a good idea, though until that happens, persuading the author or thread submitter might be a community-led solution (and we can always move them, as we currently do, if that doesn't work).

Review Forums - good point. I'll note that the 'scouted' flag only be visible to Scouts.

smiley - cheers


more thoughts

Post 3

Martin Harper

Fair enough - I was brainstorming to a large extent smiley - smiley

One thing that would be nice - if somebody else submits my entry somewhere, I'd like to be automatically subscribed to the resulting thread - that way I'll know about it...


more thoughts

Post 4

Mark Moxon

That sounds like a good idea - either that, or a specific auto-posting to your Space might work. I'll add it right away!


Key: Complain about this post

Write an Entry

"The Hitchhiker's Guide to the Galaxy is a wholly remarkable book. It has been compiled and recompiled many times and under many different editorships. It contains contributions from countless numbers of travellers and researchers."

Write an entry
Read more