h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

(Skip this, Job. It's just another assignment rant.)
I was going to call this entry "D-day", and use it as a form of online countdown to the assignment supposedly being handed in at 5pm today... but the lecturor just gave us all an extension until monday, so that's not gonna work as a subject...
The extension is gratefully received, though. I could almost feel significant amounts of stress evaporating from the room when the lecturor told us.

So the story so far (in case anyone genuinely doesn't know already), is that I've been working pretty much continuously for the last ...two? weeks. (It's getting hard to tell, as you'll soon realise.)

It took me the first week or so to be able to conceptualise the problem, to figure out exactly what the hell I needed to do. I spent a day taking apart an existing class which was similar to the one I needed to write.
I spent another day trying to work out which bits I needed to remove from it, apart from the bits I'd already taken out.
I spent another half-day on this, and then had a panic attack because I didn't really know what I was doing, and just thrashing (search engine term for picking random things in the hope that one of them might give a better clue of what to look for). So I took a break. And then went back to it. And then had another panic attack. So I went home. And then I had a nervous breakdown.
And then I cleaned my house, because it was more fun than thinking about my assignment. I even did the dishes, and I /loathe/ doing dishes. But it was still more fun than thinking about the assignment.

Then it was saturday, and I was planning to go in and work on assignments more, but I got kidnapped by S and spent the day being miserable at her instead. Same on sunday. Sorry S.

This week(?), I had an epiphany about what to write, so I spent half a day removing the bits from the class I'd butchered that I didn't need, and changing the rest to fit the data I was going to use it with. That took about nine hours of continuous work.
The next day, I wrote extra bits for the class, to make it into a theoretically-working object. 12 hours.
Next day (wednesday), I bugtested. Not all bugs: just the compile errors. The runtime errors were thursday's job. Combined: about 26-28 hours' work.
Actually, I can't remember which day it is, but I'm pretty sure I've got it wrong here somewhere. At the end of the day I'm calling thursday, I had a program which does what I want for a while, and then crashed spectacularly in a ball of flame. After a couple of hours trying to figure out where/why/when the problem occurs which does this, I gave up and went home.
The day I call friday (not the actual friday, since that's today - I only know 'coz I turned off my alarm this morning... god I hate that thing.) I tried a couple of things which I'd thought of as soon as I got home and into bed (typical) the night before. You know how it is when you pull a loose thread on a jersey....
By friday night, I had only two errors left (so it seemed), and one of them was probably causing the other as far as I could tell, so I really only had one error. But by this point I'd put in an incredible amount of effort and time into this project, and I hadn't spent more than twenty consecutive minutes at home and awake, in over a week and a half. Plus I still wasn't sleeping very well, and was far more than just dog tired.
So I was unable to figure out the problem, and decided to see if the lecturor could help. I went to his room, but there's a note on his door which says "I'm in G.1.07" So I tootle down to G.1.07 but he's not there either. The department secretary hasn't seen him, and nobody else knows where he is.
I try to resort to e-mailing him, which I was hoping to avoid since the nature of the problem was one that needed to be dealt with in person, but the school's webmail system had crashed.

So on "saturday", I went to a lecture, went to a lab, and then went to find the lecturor again... the note was still there, but at least this time the lecturor was in what is aparrantly his new office, in G.1.07
Talked to him, and although he wasn't able to help specifically, while describing the structure of the program he pointed out a couple of things I could be doing better, one of which was quite major.
I figured by cleaning the redundancy he pointed out, I might be able to figure out where the other errors were coming from... actually now that I think about it it might just have done that. I'd also noticed a couple of other things which could be optimised, so now my code is /very/ clean, /very/ fast, and still doesn't work.
Luckily, I noticed a slight glitch in the output which gave me a hint of a small something wrong... which when I traced the thread far enough, turned out to be quite major; my main formulae was entirely the wrong one to use.
Today, I'm hoping that if I clean the (new and freshly altered) code a bit more, I can figure out why the program won't run for more than a couple of seconds now: it seems to be the same problem as I had four days ago but just making everything crash sooner. Stink.

Still, at least I now have the weekend to work on it. As well as the /other/ assignment due in on monday.

As my posts to this journal go on, I imagine they're going to get more complicated. If you're already bored, you should unsubscribe now. And remember to unsubscribe to any other journals as soon as you see me tell Job not to bother reading them, too.

If you're not bored, then you're weird.

And if you're not weird, then what the hell are you doing reading my journals in the first place?

Oh, and I've managed to get at least four hours sleep a night for the last two (three maybe?) nights, and I'm throwing as many vitamins as I can stand down my throat, so I'm feeling heaps healthier than I was a few days ago. Stop worrying! I tend to look after myself, wherever possible...

Ye gods I talk too much. I wish one of the voices in my head would tell me to shut the hell up.

jerms! you most certainly do not talk too much! ! and i'm soo glad no little voice in your head tells you to shut up!

i'm obviously not going to unsubscribe from this conversation........ and thank you! yes i am weird!

glad you're taking care of yourself

Well taking care of yourself as well as you can anyway.

Well I've just spent the last several hours trying to fix my informational loss function. I went to see the lecturor when I was on my way to go get food, to try to figure out what I was doing wrong... previously some parts of my data were being set to the value 'NaN', which aparrantly stands for 'Not a Number', and I figure is equivalent to a divide-by-zero error. But it turns out the function was slightly different to what I had; in some ways it was simpler, but in other ways it's more complex.
After playing with it for ages, I've now got this section of code:

...



Okay never mind. I can't copy from that text editor and paste it into this page. You'll just have to take my word that it's: a) an extremely mathematically complex algorithm, and b) a beautifully elegant piece of code. There are no wasted computations, and no double-handling of any data. Figjam.

So what the function as a whole does is look at all possible splitpoints of a dataset, for all attributes, and compute the potential information gain of splitting the data at that point. The infogain formula is simply (entropy_before_splitting - entropy_after_splitting). Unfortunately, the function currently returns the splitpoint with the minimum entropy (because that's the best splitpoint). It doesn't actually work out the entropy before splitting the data, because I can't figure out how to get that down to one single number. I can compute the entropy before splitting for each classification value, but then what?

If I can figure this out, I think I'll have a working program.

Well...........Can't you just add the individual entropy values together?

By sheer coincidence, that's exactly what I had to do. I can't just guess these things, though. That's how I keep failing algebra, I think.

Well I wasn't guessing........ You said that you got multiple entropy values and needed to have only one, and by the way you said it it just sounded like that was the way to solve the problem.

That's the obvious choice, yes, but that doesn't mean it's the actual answer. It might have been the product, for instance. Or the dot-product.

Anyway, that error's resolved now. And I've got another one instead.

Why the hell is it returning an infoGain of >0 if it can't actually perform a split on the data?!!!

My face right now looks something like this...

AND I read your journal. Is it any wonder I dont read long posts when I end up looking like this ???

I /told/ you not to bother!

Man, my code takes a long time to run when it's printing out all the data which passes through it.

....

Oh my god it's still going.

...

Oops. It's printing out all of the data, every time it looks at any one part of it. I feel foolish now.

Okay now I know why the infogain is > 0 and it can't split the data.

The splitpoint is supposed to be halfway between two attribute values, but unfortunately those values aren't always distinct, making the splitpoint the same value as both of them. And since the splitting function splits the data into instances <= splitpoint, and instances > splitpoint, it puts all of the attributes with the same value into the left hand side. It wouldn't be doing this if all of the data was something like 23.3462385929836673 or something. Instead, it's all 3,4,4,4,5,5,5,5,5,6,6.... etc.
So how the hell do I fix the problem?

The best way is to force it to choose another attribute to split upon. But that ruins the whole purpose of going through all of the attributes and all of the possible splitpoints, in order to find the one with the best infogain.

Grr.

Maybe I'll try it, just to see if it works.

Well it doesn't crash. This is good. Maybe I can just use that bit as a haxor. I'll probably lose marks, but I don't care any more.

Time to clean and optimise.

yay!
ok, so i dont completely understand.... but hey.. you used and you sound kind of positive... so i'm thinking this is probably good news of some sort... in wich case.... congrats]

Probably too late now but you could...
a) Lie
b) Change the data
c) Ignore Me!