h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

I spent the fateful evening at the office

Now, a lot of people will say "bummer! There was no Y2k bug.". Well, I'm here to tell you that there was a y2k bug, and I'm also here to tell you why the end of the world as we know it DIDN'T happen.

I work for a healthcare group (44 hospitals). Our preparation started more than a year before 31/12/1999.

We formed teams that started off looking at what kind of "smart" devices we had (you'd be surprised how many of these things the average hospital has). Then we evaluated these to find out if they had any kind of date dependent functionality.

Again, you'd be surprised. Some measuring devices keep track of when they were last calibrated, and will shut down if they have run too long without calibration. What were those devices going to do come January 1st 2000?

We also had to plan for a worst case scenario. What if there was no electricity? What if the telephone network went down? What if the cell phone networks went down? What if the water supply was interrupted?

See... we had to be careful for two reasons.
1) We run hospitals. We had to take all possible precautions to ensure the well being of our patients.
2) (all companies were faced with this one) The lawyers told us that there was no way out. Once the press had got hold of the y2k story you couldn't argue that you didn't know about it, and so you HAD to take precautions to protect all aspects of your business, and you had to be able to DEMONSTRATE that you took the problem seriously.

That's why the bug never bit. Business HAD to get it's house in order. That included making provisions for the fact that nobody else might have got their house in order.

We beefed up the generators at our hospitals. We arranged for extra diesel to be at hand in case the generators had to be kept running for extended periods. We even put special emergency water tanks at some hospitals (we had two hospitals in Jo'burg that could go a week without external electricity and water supplies - in fact we still do have them). We ruled that no elective surgery would be scheduled two weeks either side of the new year.

We checked EVERY PC in the organisation (don't ask). We fixed the ones that could be fixed, threw out the ones that couldn't, put stickers on them all as we checked them.

I personally went through thousands? millions? I don't know how many lines of computer code and looked for calculations or logic that involved dates. When I found such code I went over it with a fine tooth comb to see if the code would work as required after December 31st 1999. Yes! We found y2k bugs in our software.

Then we created a whole imaginary hospital - but with real hardware and the remediated software.

We then set up a test model that spanned the end of our fiscal 1999, the 1999/2000 change over, the end of the tax year, right through to the end of 2000.

We isolated this system from our network (so we could set the dates on the computers ahead of real time) and captured the data speficied in the test model. We ran day-end processing at the end of each "test day", month-end processing at the end of each "test month" and so on. We took backups all the time so that if we hit a bug we could roll back to the state we were in before the bug hit, fix the bug and repeat the processing.

After 2 months of this we had a properly remediated system. We boxed all our test data, before and after program listings, bug reports and back up tapes in case we had to produce evidence in court.

But wait! There's more...

We then had to incorporate all the changes made to the live system while we were testing and re-test all the effected programs.

Then we had to roll the remediated system out to all 44 hospitals.

This was the trickiest part of the operation. The logistics took some managing.

And just before December 31st we took a live system and double checked the day and month end processing and that the date rolled over correctly (which it did).

Then came the fateful night. And it showed how you can never cover ALL the bases.

We had decided that a quorum of the board had to be available in case important decisions had to be made. We also ruled that because we hadn't had a guarantee that the phone network would stay up that everybody working that night should have a cell phone. And not all cell phones would use the same network because one of the cell networks could go down.

But what if BOTH cell networks and the land lines went down? Hmmmm... great idea! Get radio hams at all the hospitals. We got keen, capable volunteers to man radios, which were linked to generators, at the hospitals. Now we could DEFINITELY communicate no matter what.

Wrong!

Because 31/12/1999 was a period of high sunspot activity - and that caused problems for the hams...

But it seems everybody else's preparations had been as thorough as ours. Shortly after midnight we checked the network. Still up. Power? Still up. Water? Still on? etc. etc. etc. We connected to all our remote systems - still running, date correct etc etc.

We repeated the tests at 6am. Everything still OK. We went outside and had a big barbecue breakfast.

We finally stood down at mid-day (though we remained on standby).

I got home that afternoon, turned on the radio and there was some big-mouth on the air telling Jo'burg that the whole thing had been a hoax. That IT companies had put the story around so they could earn themselves lots of extra money fixing a "bug" that was never there.

I phoned up the station and had words with him.

Then I took a sleeping pill.

Well, thanks for the work. I reckon that it's a truism that if you're doing really good work, no one will (or should) notice.

You know, you might want to post that as an article. I think it's pretty good.

Uh... "post as an article". I thought that's what I did.

Please enlighten.

TIA

I work for an IT company. We design critical safety systems for Oil and Gas companies. We did all the Y2k checks (for a reasonable price of course!) starting in early '99. We spent months working on checks and fixes for clients. I would guess half of our in-house computers were known to be Y2k 'dodgy'. When i asked the Development Manager what our own policy was, he replied we'd see what happened when we came back to work after New Year !
Made me wonder wether we were really taking it seriously or not. As it turned out, our clients had no problems and neither did we. Was it luck or money well spent ?

The future never arrived. The true oddity of Y2k was the realization by many procurement managers that you had to really know something about computers to buy them intelligently, network them and keep them running. Shades of the old IBM days. Grace Hopper knew there would be a possibility of a problem. So did Billy Gates and Steven Jobs. The wiz-kids at MIT knew. So did the folks at PARC. H**l, Turing probably knew about it, or guessed it. God Bless Real Systems Engineers and a pox on the bean-counters.