Posted: 26^th October 2009

Robots Exterminated

If you haven't been paying very close attention, you may have missed the news that a small but significant change was made to h2g2 last week.

There used to be a 'tag' on all h2g2 pages, which told the software that drives search engines not to follow any of the links it found on the page. Any page that wasn't an Edited Entry also had another tag, which told the search 'spiders' not to index the page - effectively saying to search engines 'forget this page exists'. This meant that, effectively, most of the site was invisible to popular search engines such as Google and Yahoo!. A webpage's prominence in search results is decided by a number of factors, and one of these is the number of times other webpages link to it. h2g2 was shooting itself in the foot by telling search engines to ignore those thousands of links we have flowing through the site.

And now, the tag has gone, removed from all BBC messageboards to make them more visible. I'm delighted about this. It's something I've often raised, along with TRiG and Psycorp, as an easy and potentially very effective way of raising the site's profile. It's going to take a while before the spiders find every h2g2 page (there's an awful lot of them - if anyone can put an actual figure on how many pages h2g2 contains, I'd love to know) and we can't expect to see instant results, but it will be interesting to see just how quickly Google indexes the site. At this moment, there are 350,000 h2g2 pages indexed on Google, 34,400 of which are conversations that would not have appeared in search results in the past.

Don't forget, we're still on the look-out for Halloween-themed articles, poems, cartoons and stories for our next issue; please try to get any submissions in by this coming Thursday if you'd like them to be included.

Rich

• 
  Platypus Dancing
  


Meanwhile, on the edge of utter silliness...

• 
  Murder on the Dancefloor
  


Whispers to my Footsteps

• 
  Toubab Travels
  


Djembe drummers and kora players

• 
  Escape Pod Dreams
  


The 'Boy Named Sue' Issue

• 
  Smudger Snippets
  


Thinking

• 
  The Legend of Fatty-Fat McGee
  


Superb poetry from Terran

• 
  Bertie and the Beast


Don’t cats always land on their feet?
Chapter 9 Part 3

• 
  Living in a Mining Town - A Memory
  


The Thinker shares her mother's memories

• 
  Mancunian Blues
  


Climb Every Mountain

• 
  Notes from Around the Sundial
  


Part Three of Gnomon's UK trip

• 
  The Acrostic Poetry Challenge
  


What did PBS come up with for 'ALONE'?

• 
  The Post 140 on Twitter
  


The 'Tweet-A-Guide-Entry' Challenge

• 
  Announcements
  


Meets, sites, events etc

• 
  Entry of the Month
  


The best of September

• 
  Email the Post Team
  


For submissions and editorial feedback

• 
  The Post Readers' Forum
  


Chat about The Post with other readers and the Team