h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

The spiders were stopped from the outset - it was always understood that conversations on here would never show up anywhere else on t'Internet. The new arrangement makes a bit of a farce of post-moderation, as by the time something "offensive" is pointed out and removed it has already been copied elsewhere, thus pushing the BBC towards a global pre-moderation policy. Not good.

>>The spiders were stopped from the outset<<

Are you sure about that? I remember it as being something that happened after I joined. I thought the spiders were causing other problems separate to publising/moderation issues.

Hmmm, maybe convos used to turn up in the h2g2 search, and that was changed?

When the BBC took over h2g2, the conversation search "broke". They said it was accidental and they were looking to fix it, but it always seemed just a little too convenient in that it happened when they were trying to post-moderate all the "legacy" threads and really didn't want people reading them until they had gone through them all first.

But from what I can remember from back in the pre-BBC days, only Guide entries could be spidered by external sites, later restricted to only Edited Guide entries.

I remember the conversation search working for some years after Rupert (bbc takeover), but not that many. It has been ages since it has worked. Since it still doesn't work, I for one welcome our new spider overlords since without them, there is no way to find anything in any threads. And since really old conversations may not even show up in one's personal space anymore, they are effectively gone down the rabbit hole.

Indeed. And because the interesting old conversations have no direct links any more, they will not be spidered. It's the worst of both worlds...

We need the internal conversation search *fixed* and conversations removed from Google's cache. Otherwise, they might as well give up on post-moderation.

Or is that the secret agenda? Do they think they'll be able to persuade the Powers that Be(TM) at the BBC to stop paying for post-moderation as it is no longer effective?

F124?thread=229950

Jim Lynn's comments in his journal suggest that the reason spiders were blocked in the first place was because of spiders, err, spidering legacy postings when they were unhidden, and thus placing a amssive strain on the servers.

Yeah, I seem to remember the spiders were stopped because they were causing huge slowdowns.

"Jim Lynn's comments in his journal"

I stand corrected. My memory isn't what it... er...em...

For some reason I remember threads discussing spiders and explaining they were disabled because of modding issues. Ironically, I can't search for them to confirm.

It sort of makes sense, as IIRC back in those days there were far fewer servers to share the load.

*But*, if the BBC wants to maintain post-moderation as a form of legal protection, it should be worried about its content being cached off-site.

It's a very sad thing that some people need to be legally protected... from words.

I am sure the spiders were kept out because they slowed the servers down. Everything said here is in the public domain - as would be so in all the other public forums that you might post to.


I was delighted yesterday to discover that something posted 5 years ago was found so easily by googling it. Ain't life fun?

>>> Ain't life fun?
Yes, indeed.

I regularly use google to search the Guide. It's a great way to find relevant information in conversations, the UG, and the EG all at once.

re-refreshcan we have the logged in "eyes" looking the other wayI'm fed up of the wallpaper at that side

I'm not going to go through it all again in detail because the removal of metatag is an incredibly dull subject, but the removal of the nofollow tags was something that I suggested a couple of years ago. I raised it in a Post column 12 months ago - A36186924 - and I think most Researchers who read about it supported it (there are more links to some relevant conversations in the 'No Subject' thread on that Entry).

The idea has worked: I've informally tracked some of my own Entries, and they are climbing the search results as more pages get indexed. As a result, h2g2 is far more visible on the wider web, so in that sense it's a positive thing.

I don't understand Peet's problem with articles potentially causing a problem because of syndication. Nobody else on the internet has bothered to republish even the first post in this conversation - it's four weeks old, a couple of clicks from the FP, and has been indexed by Google.

There are a number of things you can criticise h2g2 for, but losing the nofollow tag isn't one of them. It's actually a very brave move - anything anyone posts can now be seen by a Daily Mail reporter anywhere...

Oh, if their reading's as good as their writing, I wouldn't feel too threatened about it.

So is removal of the nofollow tags what's allowed google searches of convos? I only noticed the ability to use google to site search a few months ago, and I started a thread in Ask about it, which drew a few comments but not the excitement I expected. I do agree with Peet that letting people know about the change would have been good.

>>
Nobody else on the internet has bothered to republish even the first post in this conversation - it's four weeks old, a couple of clicks from the FP, and has been indexed by Google
<<

Does google caching, the internet archive etc count as publishing? I'm thinking the issue is to do with libel/slander, but don't know if a website can be held accountable for work it's removed but is already cached somewhere else.

"I don't understand Peet's problem with articles potentially causing a problem because of syndication."

That's because that *isn't* my problem. (And I've only ever been talking about threads, not articles...) Truth be told, I don't have a problem with it, I think it's great.

What I was drawing attention to was the BBC's paranoia about having full control of everything they publish. When the nofollow tag was in place, if someone pressed the prod-a-mod to complain about potentially libelous postings all they had to do was hide the post and they were covered. Now, though, by the time somebody notices a post that could get them in trouble it could have been copied to several other sites by any number of spiders.

Like I say, *I* don't have a problem with that - it's how the web is *supposed* to work! But the BBC may have a different viewpoint, and the thing that worries me is that it will make them less inclined to allow DNA sites to be post-moderated.

Isn't it fair to say that if the BBC removes the offensive/copyright breaching material from the site itself, it will have done enough to avoid any legal action?

It might be, if it wasn't for the fact that any cached page with a libelous post on it also will have "© BBC" plastered along the bottom.

Yes, but if it is "© BBC", what right do any caches or aggregators have to copy it? Surely a quiet word would put a stop to it.

But it also typically has "cached" along the top. I think this is a non-issue.