h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

This journal is just a sketchpad for the current rant that I need to get off my chest. Well same as usual I guess, but this time I don't expect it to be constructive at all.
Incedentally, this journal is brought to you by the letter: 'Aargh!'

I've just spent most of the day (literally) trying to get the code for my current assignment to compile. Every time I think I've fixed it, I recompile and... one error. Only one error. I don't think I've had a second error in the same compile since about 2pm (it's now almost 8:30pm).
Admittedly it hasn't been the same error - it's been different almost every time - but it's still one error.

So I was most shocked when it compiled! About 6pm, I think it was.

Of course, it didn't /work/. I've just moved on to run-time errors instead of compile errors. But it's still nice to be able to play with the output and have functions which can do my bug-testing for me and so on.

Most of the problem I've had has been with interfacing with the WEKA package that the machine learning group has spent the last five years odd working on. Considering that one of the main programmers was the guy who was my lecturor in object-oriented programming techniques, the code sure has some terrible object-oriented programming techniques.
The use of set and get functions has been erratic at best. The copy function is missing in one of the major objects, and is replaced instead with a different type of constructor. (?!!) There are random errors without reasonable explanation whenever I try to alter the attributes of an instance in a way which is apparantly not the way they want them altered. And so on.
I wonder whether these guys have actually looked seriously at the HCI issues of their package. Probably not.

And now I find that after spending about a week trying to convert the raw data into something that their package can make use of, I then had to remove the classification attribute so that it didn't get converted into word vectors, and then put it back again afterwards so that the cross-fold validation can train itself with any sort of accuracy. But between the conversion to word vectors and the cross-fold validation I also have to run the data through an SVM filter, which breaks if there's any nominal values in the instances object. Including the classification attribute values.
At the moment my program is crashing between the STWV conversion and the SVM filter, because for some reason if I have two files as datasets then one of them will have a classification attribute and the other one won't. I can't work out whether one of them has had it put back too early, or the other one has had it put back too late. I can't work out which one is the anomoly. And I can't work out how I can remove the classification attribute from the dataset for both processes, and still have it available to reinsert after filtering. It would be fine, but for some reason the WEKA package comes up with an error if I try to add an attribute object to an instances object. And trying to store the attribute object won't store the values for that attribute anyway. Which means I need to keep a copy of the original pre-converted dataset so that I can copy the attribute list across. But that means that I have more than 2x21 entire novels sitting in memory, which breaks the STWV conversion function. Without warning, of course. I'd hate to think what it would do to the SVM filter.

Once I've figured out what's happening with all these problems, and what I need to do to get around them, then I have to find a way of reordering the word vectors into a frequency list to be able to extract the most significant features of the texts.
Actually I wonder if the SVM filter can do that for me?
I'll find out when it starts working I guess.

And then I have to find a way to remove the most and least significant features programmatically.

And then I have to graph the thingumee ratio. Ten datasets for each block of text; which makes about 47,000 points of data. Hmm. Perhaps I shouldn't do that by hand.

Bugger.

awww.....



i didnt understand all of that...... well lots of it i didnt understand.

but here i am offerig my company, support, s, and s.

ooooh.......
i just realized....... it says 'please ignore this' in the title.... and here i am posting on the forbidden thread....
liveing life on the edge arent i?

ahh.... good.... hes laughing... does that mean that i dont have to start sneaking away from the conversation, trying in vain to pretend that i wasnt here at all?

Aargh! It's not working!

The classification attribute has the values reinserted for all instances except where those instances relate to the first file added to the dataset. How the hell does that make sense?
The code to reinsert the classification values is fine. There's nothing wrong with it. I narrowed it down to the line: newData.instance(i).setValue(newColumn, setToValue); but all the arguments are perfect, and all the return values are perfect. The attribute object exists for all the instances, and it's not coming up with an error so that's not the problem anyway.

Is this supposed to be a feature?
No, wait. I'm adding the values explicitly. It can't be a feature.


(I'm just using this thread to rant on about my code; I don't expect anyone else to follow it or to care. You can stay if you want, or you can leave if you want. I just expect most people who have been accidentally subscribed will want to unsubscribe before it goes too far, is all.)

'course i'm staying.... and of course i'm following..... well following as best i can

"How the hell does that make sense?"
no.... it doesnt.... no sense at all...*shakes head* although that may have to do with the fact i dont really understand....

its alright jerms
you can do it!

The only thing I can come up with is that it's some sort of artifact of the way the attribute objects are displaying themselves, and that the data has been there all along.

Which means I've just wasted about three-quarters of an hour.

aww....


take a deep breath....
here.... have a nice cold....

I didn't understand any of that!

Aargh! What are you doing here?! You're not supposed to wait until I'm just about to log off before you log on!

well.........surely what i said made sence....

I've been logged on for ages, I just didn't reply to anything.

Mwa ha ha ha ha.

Thats what happens when you live in a country exactly 12 hours time difference from the one I'm living in!

exactly 12?
hmmm......
*trys to calculate it out...... but being 2:57am its not that easy*

mmmmmmmm............ my mind is mush..... ah. whatever i'll think about it in the morning. oh! no! wait! it is the morning! i think about it in the afternoon then......when i get up.... if i ever go to sleep...

I was talking to Jerms since I don't live in Russia!

i know i know.......
i didnt know you were both 12 hours apart... i was going to work it out for myself to see if its true, not that i dont believe you. but my brain doesnt want to think about it.

It's either 12 or 11. But since I was online at 10 and you said it was 10 there I would say it was 12 hours difference.

Either that or he's living in London.

jerms!? are you living in london and not telling us!!??


well now that i've been to sleep and am awake again.... yes..... that sounds about right......

Nah Im living in Noisyland.

and you didnt tell me!?
....how will you birthday present ever get to you if i'm sending it to the wrong country?