h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

OK, this should probably be in "Community Soapbox" or something, but Askh2g2 is where most people hang out, I think, and also this is in the style of a question.

First: background. Yesterday the Editors Choice Edited Entry on the Front Page was an entry titled "Console Wars", which purported to be about the history of the "battles" between manufacturers of videogame consoles. I was quite interested, and read it.

It was a curate's egg - parts of it were excellent. The sections dealing with the console from the Nintendo NES onwards were very good. But there were several problems.



BORING TECHNICAL BIT - IF you don't care THEN GOTO next popcorn because you won't miss much ELSE read on

Although it was titled "Console Wars", it didn't even MENTION the first console war - that between the original Atari VCS and the Mattel Intellivision.

It gave the impression that the home videogame boom started with personal computers such as the Spectrum and C64, followed by the Magnavox Odyssey then Atari VCS. In reality, it was the other way round.

It gave the impression that the Atari was more powerful than the Spectrum or C64, which was the opposite of the truth.

It failed to mention the Colecovision, an important console because it saw the first appearance on a home machine of Mario, Nintendo's mascot.

It completely failed to mention the way that home computers killed off the first generation of consoles, and the ironic fact that it was Nintendo, with the NES, which in turn killed off home computers.

END BORING TECHNICAL BIT



These errors were present despite the entry's passage through PR, and despite the presence on the list of contributors of at least one person who I would have credited with a good knowledge of this aspect of recent history.

The entry was picked by a Scout - it was subedited, Edited, and picked for Editors Choice despite glaring factual inaccuracies about history so recent that I, who am not exactly ready for my bus pass, remember it clearly from my adolescence. The facts I've listed can be easily verified - retro-gaming is HUGE on the web, there are hundreds of sites where this history could have been checked.



OK, I'm NOT blaming the author. He wrote an entry which was clearly intended to be about the console wars he knew about - Nintendo/Sega, Nintendo/Sony, Sony/Micro$oft. That his older history was in error and that the title was slightly misleading is not at issue. What is at issue is the process the site put his entry through.

The process of Peer Review failed him. The process of picking failed him. The process of subediting failed him. And to compound the problem, the entry was made Editors' Choice, drawing more attention to it.



I'm in an uncomfortable position. Previously when I've seen a problem with the way the site works, I haven't faffed about complaining, I've just offered a solution. Well, this time I'm a bit stuck. I don't know what to do. All I know is that I'm sure *something* needs doing, because the current system isn't good enough.

I appreciate that the conversation threads below an entry can be a place where you can get errors corrected. But I strongly feel that by the time an entry gets to the Front Page, and *especially* if it's going to be Editors Choice, it should at a very minimum be factually accurate. The Front Page is where visitors, and potential new members, go first - so what's there should be the best it can be.

So - the question.

How can this be achieved, sensibly?



Can we expect subeds to do it? Nope. Too much work, I think.

PR clearly doesn't work, if errors this glaring can get through.

Are the Scouts picking too early, or not carefully enough? Should we expect a Scout to check an entry for factual accuracy before recommending it? I'm sort of thinking as I type here... I quite like this one. Of course, the author would have to be on hand... I'm envisaging a Scout going to the author's space and saying "I wanna pick this, but I have to be sure it's all accurate - gimme some links so I can check". Put the onus on the author to show that it's accurate, but don't let the Scout pick it until they've checked.

Too much work? Possibly.

Peer Review needs better quality control. Anyone got a better idea?

H.

F118077?thread=269841

Please note especially post 20, where I note that many of the errors WERE brought up in the Peer Review thread, but still were not corrected.

How can we stop that happening again?

H.

I know how you feel. From both ends of the spectrum. I looked at the upcoming entry on Saunas, and it failed to mention the Roman saunas, to name one thing (And I completely missed out on FoxTrot in PR ). In my EG entry on the Statue of Liberty, Bels brought up a number of points that were not mentioned in PR that I would have been happy to put in.

There isn't much of a way to stop this. Perhaps if you think a recommended entry is incomplete, or is structurally flawed, you could say something to the Italics. But they don't like to add things, just correct them.

Maybe we can stop this problem with the scouts though. For one thing, it might be a good idea for scouts to scan the PR thread and see what has been failed to be included. But with Update HQ on the wane, largely replaced by Editorial Feedback, the content of entries is pretty much set in stone as soon as an italic clicks that big shiny button to accept it. It seems.

Do you have anything specific in mind? I'm just giving my

I just reread a bit in your post (I sort of duplicated it in my last post. You said-

>>Are the Scouts picking too early, or not carefully enough? Should we expect a Scout to check an entry for factual accuracy before recommending it? I'm sort of thinking as I type here... I quite like this one. Of course, the author would have to be on hand... I'm envisaging a Scout going to the author's space and saying "I wanna pick this, but I have to be sure it's all accurate - gimme some links so I can check". Put the onus on the author to show that it's accurate, but don't let the Scout pick it until they've checked.

The scout - as part of their responsibility - shouldn't have to do that. It's not the scout's responsibility, it's not the sub-eds, or the italics. It's the author and partially the PR responders. I've noticed that not many entries have been recommended without some feedback. When it hasn't had much feedback or nods of approval, an italic will stop by and ask if it's ready. This is good. I was completely taken by surprise a few weeks ago when an entry of mine was recommended. In my opinion, it wasn't ready and gave too much work for the subed.

But alternatively, if there's too much stalling, good entries can be lost. An entry of mine was recommended, but a friendly PR'er popped in with some suggestions and Jim freed up the scout. I incorporated the suggestions to the suggestor's satisfaction and that entry seems destined to be lost in PR

If there's a more thorough process, it could help quality, with the author, PR, scouts, italics, subed and then italics again checking it for quality. But it may lose entries.

>>Too much work? Possibly.

Too much work? Clearly for volunteers.

hopes we don't simulpost...

Lets look at each step of the process to see where we can improve.

Lets see where things can fall down.

Imagine I write an entry with technical inaccuracy in it.

1. Entry is entered into Peer Review.

Hopefully someone will comment who is familiar with the subject and I will correct it.

2. Scout picks entry.

You can't expect the scout to know that there are glaring errors in a piece if it is on a area he/she/it knows nothing about.

Hopefully the scout won't pick the entry unless I've corrected all the details on it, however the scout may feel that the errors are small, that the authour has left the buidling and it would be a pity to loose the entry to the Edited Guide.

3. Italic approved the entry

They are unlikely to notice inaccuracies, however they should notice if anything has been pointed out in the Peer Review thread and not been acted on. This happens quiet a lot

4 Entry goes to a sub editor

Again you can't expect them to be an expert on the topic, however they should check the PR thread

5 Entry again gets checked by an editor before it goes on the front page.

I know from experience that this happens!



Factual innaccuracies can get through in two ways.

Firstly it may not be raised in PR

or it may be raised and not acted on.

The solution to the first one is, IMHO, simple, we all need to spend more time in Peer Review commenting on entries.

the solution to the second one, is difficult, however good all systems are they are going to fail occasionally, we have an entry being checked at quiet a lot of stages.

I don't like the idea of having to provide links to prove that your stuff is factually accurate. That way the only stuff that can be published on h2g2 is stuff that is already somewhere else on the web. Much of the stuff I put in comes from my head or from encyclopaedias and books. Some of it directly contradicts a lot of the stuff published on the web, because let's face it, there is a great skill in interpreting the Web and telling whether what you read there is reasonable or not. There's plenty on it that is just plain wrong, as H himself knows well.

It would be nice if scouts only picked entries that they knew something about ... that way they could do a quick check for accuracy. But many of my entries were about subjects that I wouldn't expect any scout to know abou. I only learnt about them a few weeks before writing the entry.

I don't know what to do about the problem outlined by Hoovooloo. But I don't consider it a major problem. In general, the entries that I know about are usually fairly accurate by the time they get to the front page. (I can't comment on the rest).

I have written on h2g2 article for which I have provided scientific style references, but I don'tthink it is possibe to expect people who haven't been in higher education to provdid references.

I know of on reseacher who does provide references, incidenly I advise authours to remove them unless they are links to sites that could provide futher reading.

Good points Gnomon and Z. But I find it harder and harder to look up and check facts for PR. I consider this the writer's responsibilty. Occasionally, if the author poses a question about one area, I'll look it up and make a comment, but other than that, I trust an author.

Luckily, entries that are good enough for the edited guide tend to have good authors and less problems. We're talking about a small percentage of entries that are incomplete or inaccurate, though I come across 2 paragraph explanations of topics worthy of 200 paragraphs all the time...

Well I would hope that you wouldn't have to check all facts, however if it was on a topic that you knew a lot about, or even on that your knew quieta lot about you would know that the facts were correct.

Poorly worded. But if it's an obscure topic, PR has little chance to improve it, under that idea. I'm still waiting for someone to propose a fairly obvious modification to one of my obscure entries that I don't know how to do

I smell a "PR Watchdogs" club

This is the third time I've tried to post. I deleted the others. I substantially agree with hoovooloo that there is something wrong with the EG process. However, I think the scope of the problem is wider than Peer Review. I disagree with so much of the current system as it appears to be implemented (though not the ideal of it) that it is hard to post on this topic.

Is there something you have better in mind?

Surely this system is better than the old way of putting it in a long queue for approval? I honestly can't figure out a better system. Can you give me a quick example?

Erm, re-arranging that a bit, yes and no. It's not the ideal/principle but the details. So although the EG concept is nice, it's the way it is run in practice that I see as a problem.

I didn't see "the old way" in operation. Maybe it _was_ better.

I don't think I can give you a quick example of any part. It would probably turn into a rant.

What kind of details and practices do you see as a problem? I'm genuinely curious

I didn't see the old one either, but reading about it, this sounds better anyway.

I have found one particular entry that's been recommended and gone through the whole process and is listed as being "returned from sub-editor" but it doesn't have a "pending approval" label on it. It's been sitting there for weeks. I didn't notice it until after it had been recommended and I was quite sadly surprised by the (low) level of scholarship in the entry. But then, it was on a subject that is complicated. The subject could have been covered in a fairly simple introductory way, but the researcher had touched on some issues that needed a better treatment if they were going to be included. I made a note to the entry as soon as I found it and then another to the recommended version but nothing has changed. And the entry seems to be sitting in limbo.

wow. I guess that was my long-winded way of saying, "yes, I've had a similar experience." sorry.

I'm somewhat of two minds about this issue, Hoovooloo. I agree that it would be great to have the edited entries as perfect as possible: that's why I went to the trouble of writing a new (long-winded) Canada entry. And it had a factual error by the time it hit the front page as an edited entry. It was noticed right away and an italic fixed it within minutes, while it was still on the front page.

On the other hand, the guide that this site was meant to model contained much that was wildly inaccurate. I'm not sure that we actually want to get too close to perfect even if we could. Part of the charm of the original Guide was that it didn't succeed in being accurate.

I definitely think that we have to all do what we can on Peer Review, and it is, I think, the author's responsibility to verify his/her facts. We can't necessarily expect commentators in Peer Review to always know the subject of an entry as well as the author, or even at all. I have an entry in Peer Review right now and I've never yet met anyone else, here or in the real world, who has even heard of the subject. I've tried to be meticulous in my facts and I've included references if anyone wants to follow them up. But in the end, the reader pretty much has to trust me.

I guess I'm saying that, yes, the system is a little ill-designed, like democracy or the human knee, but fiddling around with it may not make it any better and may make it worse. We need to do our work as peer reviewers as well as we can.

Anhaga sparked an interesting point. The most thorough entries are more prone to have a few errors. On my seinfeld entry, which, by all accounts is pretty thorough, I make a very small reference to Superman, and immediately a Superman Fan corrected an error and it was fixed before I could say Kal-Eel.

These little things are obviously easy to take care of, posting it on the entry or editorial feedback will fix small problems. Incompleteness and a large inaccuracy is hard.

If a large inaccuracy is spotted after an entry makes it to the front page, what is the procedure? Does an Italic rewrite it? Curious...

"Kal-Eel"

Isn't that supposed to be "Kal-El"?



As another example (actually, to be accurate, the same example in another context, sort of.) the edited entry on Canada was generally agreed by Canadian researchers and the italics to be woefully inadequate. I simply wrote a new entry, including a few facts from the original (hockey) and giving credit to the original researcher. As I understood it, the new entry would go through the usual Peer Review process and thenbe quietly slipped into the edited guide. I guess somebody slipped up and made it the editor's choice (yet one more error).

So, in the long run, that's probably the route for big inaccuracies or inadequacies.

(did I mention Superman was created by a Canadian?)

Any system can break down from time to time and I reckon that the 'Editorial Process' broke down in this case...

I don't know how much wailing and teeth-gnashing I'm inclined to do over this. If/when a formal updating procedure is in place, it *should* be easy to fix stuff like this. Under the current system, you can propose an update, write it and sometime before the end of time, I might get to it.

Let me pose some questions that all have the same answer:
- Do Scouts sometimes pick entries that I wouldn't pick?
- Do entries appear on the front page that I wouldn't have edited?
- Does stuff get in the Edited Guide now that I would have returned to the editors back when Subeditors were the sole authority?
- Are my standards different from other people?

And my answer to all four is:
Yes.

No system is going to be perfect. But I'm pretty comfortable with what's going on here both with Scouts, Subeditors and TPTB.

I've read some interesting stuff that I might not have otherwise because somebody else felt the topic was EG-worthy. That's part of the site's appeal to me.

Back in the days of the Critique Fun Run, Vegiman made it a rule that an author submitting an entry into the Run had to review all the other entries in that round as well. I don't think that was a bad thing as it encouraged comments...

But there would be ways to 'game' a system like this in our version of PR so I don't know how useful that would be.

In a case where the Subeditor and Italics missed comments in the PR thread, I guess everybody's human (with the possible exception of Ashley ).

Good points. But like you, I'm not sure what a good solution would be except to urge everyone to me a bit more thorough...


- Jimi X

Eel... Oh dear, I'm going to be mobbed for that mistake...

Rewrites? Incidentally, I'm rewriting an entry on Calvin and Hobbes. Not only is the original EG article incomplete, it's factually inadequate according to something the cartoonist himself wrote.