h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

oh
thanks BC, thats awefully helpful of you

would you like a slice of ?

thankyou Arisztid
*phew, think i got away with that *

anytime BC

*course he did*

Can I have some ?

Whats that writing in the smoke coming from the incinerator?

I'm not sure.......Maybe the wind will blow the smoke into the proper message.......

Yes it says.......

*switches industrial duty fan on blowing smoke away before it has a chance to do anything*

awww...... BC......
i was curious to know what it said....

*Wanders in*
Hey... who did that?

hmm?
who did what?
turned on the fan? it /was/ getting a little smoky....

*Looks sideways at Aristz. Suspiciously.*

No, I meant who did that? *points*

Going back to the point of the conversation (probably temporarily, but meh) : This is a copy of an e-mail I sent to the lecturor at about twenty to three this afternoon. The assignment is due in at five pm today.

Hi Eibe, I know it's getting late to be asking this, but I was hoping to at least have some degredation curves to hand in.
Unfortunately I can't seem to make the SMO object return anything other than 0.0 for the weights, which of course makes the Evaluation object return 100% correct for all iterations.

With a bit of basic bugchecking I've found out that it returns 0 because the third dimension of the arrays returned by smo.sparseWeights() and smo.sparseIndices() are all of length 0 - that is, sparseIndices[0][1].length == 0 and sparseWeights[0][1].length == 0

I have no idea why this might be. Any last-minute ideas?

Also I've noticed (along with other students, it seems) that with the
number of significant attributes given to the StringToWordVector set to 250, I'm being given back a list of over 12,000 attributes in some
Instances. With the number set to 2, it returns anywhere between 12 and 18. I can only assume that the number of significant attributes is counted as the most significant attributes for that Instance object, and the list is then added together to the other lists produced for the other Instance objects, in that Instances object.
Except that even with Instances objects with only two Instance objects, and the number of significant attributes set to 2, it still returns more than 10 significant attributes for each Instances object.
I'm not sure how I can restrict the data to only 250(ish) significant
attributes, without degrading the quality of data horrendously. Any
thoughts on this point would be welcome, too.


Both of these problems are caused by the "professional" code we're supposed to be interfacing with. There's no documentation to support any of the experienced behaviour, I haven't had a reply from my e-mail yet, I went to see the lecturor but he was in a meeting, I went back ten minutes later and he couldn't be found. I think he's gone home.

Just quietly, I was surprised that my code even managed to do this much of the assignment. This morning it wouldn't even run a cross-fold validation, and now I only need the SMO object to return valid values for me to be able to construct degredation curves! Well done me!
However even the construction of the degredation curves is only about a quarter of the way through the assignment.

The only good thing at the moment is the fact that I was able to get a medical certificate for yesterday and the day before, even though they were public holidays. So I should be able to get an extension, even though I don't want one because that means making myself even busier later on.

Sigh. I want it to be over.

oh! well done jerms!

glad you got an extension.

"...because the third dimension of the arrays returned by smo."
why do i get this feeling that we arent in kansas anymore? or not even on earth........

sounds so.... sci-fi.... but then maybe thats just me...

*sees the post just above that one*

*looks as innocently at jerms as she can*
oooh... you mean that.....
well.... *clears throat*
er.......

What was the question?

I think that statement's more profound than you realise, Job.

The whole point of this assignment is to be able to take a book for which the author's name is unknown, and collections of books by various potential authors of that first book, and figure out which author probably wrote the book.

To do this, I:

Need to convert each text into the various features of the texts.
Need to enumerate the features in the text(s).
Need to rate the features in the texts, by order of importance.
Need to collect each of the ratings for each of the types of features, for each text.
Need to remove the most and least important features from the text, and see how the ratings change when compared to the other author's texts.
Need to collect sets of the changes of ratings.
Need to graph the sets of the changes of ratings of the rated enumerated features in the texts.
Need to print out a graph, and be able to associate author's names with curves on the graph.

Then I look at the curve with the greatest change (ie the one which is heaps lower than the others) and associate that author's name with the book.
Ta da.

And of course every silver lining has a cloud. Currently, my assignment is almost 24 hours overdue. And I'm up to step three in that process. I can't rate the features in the text, because every time my code sends the enumerated features in the text to the object which rates them, that object then returns a rating of 0.0. Every single time.

I've just spent fourty minutes in the lecturor's office trying to figure out why, and have come to the conclusion that the objects my code sends to the rating object need to be libraries, not individual books.

A brief explanation of the term 'order of magnitude' is required here, I think.
I'm sure you all know what exponentials are - like powers of two or whatever. So instead of counting 1,2,3,4... you count 1,2,4,8,16...
And I know that many of you will have had to put up with the nightmare of logarithms, which is the inverse of an exponential, so you count 1, 1/2, 1/4, 1/8...
Well to change from counting in logarithms to counting in ordinary base ten, or to change from counting in base ten to counting in exponentials is a change in order of magnitude. And there are many more than just these three orders.
If it's easier, think about in in terms of driving a car. You have the distance you're travelling, which in physics is measured in metres. You have the speed of the car, which in physics is called velocity and is measured in metres per second, or distance over time: d/t. You also have acceleration, which is a measure of how fast your speed is changing: measured in metres per second per second; velocity over time, or distance over time squared.
If you graph your progress in driving from your house to the dairy, with time at the bottom of the graph and speed on the left, you'd notice that the line on it is sloped - you don't go from a standing start to top speed instantly. Your speed increases, until you reach top speed or an intersection or whatever. In this graph the height of the line would be your speed, and the curve would be your acceleration.
It's possible to redraw the graph by differentiating it, so that acceleration is on the left (and time is still on the bottom). So now your acceleration is now the height of the line, and the curve of the line is a measure of how fast your acceleration is changing - distance over time cubed.
Yes you can keep differentiating the curve over and over again. Why you would want to, I don't know. And if you can think of a good reason I don't want to hear it, because I'm sure it's boring.

But this is an illustration of what it is I'm doing with my assignment; I'm building objects which can deal with an order of magnitude number of objects created in the previous step, and I'm doing it over and over again. But I also have to be able to change the data in ways that reveal patterns, and can do something with those pattern when I find them, before going on to the next step.
Which is why each book is broken down into objects of about 500 words; roughly 400 for each novel, and then each of those objects are collected again into a combined object, one for each book. And then each of those book objects are combined into libraries by a particular author, and then each of /those/ gets combined into a single dataset, contained by my program.
But what's making it annoying is that I have to pick particular books, or chunks of books and group them together in particular ways to analyse them; over and over again. It's not just the physical space the objects take up that I have to think about when I change orders of magnitude, it's also the complexity of the analysis. And also the run-time of my program; I don't want to be waiting forever for it to be doing this the slow way. So trying to juggle all of these things together is making my head hurt and my program break. Not always in that order, either.

Okay so that wasn't a brief explanation, but I guess I needed to rant a little more. I hope you can understand why I'm stressed. And sick, dammit. And I could do with a hug. Followed by food.

well I think I can understand what you're trying to do but I'm glad you're the one with the assignment

in the absence of RL ones have

hope that helps

*hears the mention of logs*
ack!
logs!!!!
*runs and hides*


*peaks out from under the table*

Don't think he meant those kind of logs.......

Sorry I can't be there to do anything more than this......