Tuesday, July 29, 2008

Information Theory and Memory.

You can probably file this one under 'crackpot theories' but lately I've been thinking about ideas, information, and how they spread. I am sure anyone who reads enough Neal Stephenson (link) will probably come to the same conclusions as I have, but writing it all down in an open, public, and embarrassing forum can only help deepen my understanding. Either that or I'll gain enough ridicule to stop thinking about this stuff altogether. Either way I win.

Information theory is a broad and nebulous discipline that straddles math, physics, language, and enough other subjects to be patently interesting to anyone interested in punishing themselves by reading thick books. I have yet to delve much into it beyond skimming the surface but there is one example that I read that sums up a nice little chunk of it. Say you flip a coin one thousand times, and you want to let me know, in order, whether each coin flip was heads or tails. This should be easy enough. You can just call me up on the phone, and read me your list of one thousand results.

Think about how much information that is though. That is 'Heads' or 'Tails' one thousand times. That is 5000 characters worth of information. Heads and Tails each has 5 characters. So lets represent each coin flip with a number; 0 for heads, 1 for tails. Now that is more like it. We've got 1000 coin flips and 1000 'bits' of information.

Now lets pretend your coin has some nasty crap stuck to one side of it, and now your coin almost always lands heads up. In fact, you've done the experiments, and you know that this coin will land tails up only once for every 1000 flips. I call you up and ask for the results of your latest experiment. Well, you could just go right ahead and give me a stream of 1's and 0's just like you were doing when you didn't get some nastiness all over your shiny new coin. I would have all of the information of your coin flip, but since you and I both know that there is only one tails in the whole sequence, wouldn't it be easier to just tell me where it was? You could tell me that the tails showed up in space number 33, and I wouldn't have to listen to you list off your whole big stream of numbers.

What is going on here, is you have used your predetermined knowledge of how your number set will turn out to compress your information. Before, I was listening to you spout off a list of ones and zeros for 15 minutes; now our phone conversation is about 10 seconds. You have gone from giving me 1000 bits of information to giving me the same amount of information, but in only 6 bits (33 in binary is '10 0001', the location of the one). This is the fundamental basic of information theory and the idea behind all kinds of lossless data compression that you use on your computer every day.

We know how to compress data on a computer. This is a technology that has been around for awhile now, and they have applied it to almost anything that can be stored on a computer. The compression is based on the symbols that the computer uses. There are only 2 symbols, used to store the information and the computer is made to recognize and operate on vast amounts of these symbols; but what about other storage medium. What about the human brain?

This is where the crackpot section starts and I start to speculate some stuff. These aren't completely uninformed opinions. There is a cursory understanding of the involved sciences going on, but by no means is it a deep or meaningful understanding. If you find fault with anything, please publically flog me, but understand I am speculating here.

The brain recognizes a whole different set of symbols than a computer. Through evolution, mankind has had their brains hard wired to recognize symbols according to their importance in survival and passing on our genes. We are good at remembering smells, other human faces, unique visual stimuli, subtle nuances in sounds. Our brains are wired up to the five senses, and it seems as if we store this information pretty efficiently. Sure, we are capable of using other symbols such as writing, or pictographs, or even the binary code of computers, but those are symbols that are used to abstract our senses into different symbology. When you read a word, or a series of words, it creates a picture in your mind based on your experiences. You understand the words because you can relate them to unique experiences you have had via your senses.

In the coin example, a large amount of information was stored with a very small amount of information. This was possible because the occurrence of the number 1 was so extremely rare in our set, that we could convey it in a simplified form. The symbols 1 and 0 allowed us to convey the message differently with the same symbols. We used 1 and 0 to represent a larger number rather than just heads or tails. We embedded a different symbol within our limited symbol system.

Is it possible that the brain does, or is capable of doing the same thing? We remember unique sensory experiences very well. Is the rarity of such an occurrence more easily compressed in our memories because of the same principals of information theory?

My memories are mostly of myself, other people, and food. In that order. All of the math, history, science, etc. that I have learned in school pales in comparison to the vast amount of data I have stored in my head about other people and my own sensory experiences. It is harder to remember something that didn't directly happen to you. You have to solidify the concepts out of an abstracted data set (read the words, translate them into ideas). That extra step can be a hindrance to learning because words, letters, and numerals are not the symbols our brains natively use. They use direct sensory experiences. Is it possible that sensory experiences can compress themselves in our brain because that is the symbology our brain uses natively?

(find the link to the guy who can do crazy math in his head). My speculation is that this guy has the ability to encode numbers in his head as a native data type (ie sensory input) in much the same way we would remember someone's face. (find the link to the piano synesthesia girl) I also think that because this girl can encode music in her head as more than one sensory experience it makes learning music exponentially easier for her.

What if we were to take this concept into effect every time we wanted to learn something in order to compress the knowledge into our heads. Next time you are studying for a history test, try eating a new dish you have never had before or listening to a new kind of music you are unfamiliar with that corresponds with the subject matter of the test. If you are studying for a French language exam, try cooking French foods and listening to French composers. Try studying in a new and unfamiliar location. It just may be that the unique experience you give yourself may help you compress all that dry information as a unique and rare experience.

I haven't tested my theory yet, but I'll be sure to give you a full report on how it works when I do.

Wednesday, July 9, 2008

Introductions are in order...


above: Me in a back room at CBGB's stringing my guitar.


My name is Andrew Lenox. I am a novice programmer, an ex-musician, a long time student, a husband, a fledgling game designer, and a dog person; in no particular order.

I started this blog because:
  1. I have a few far flung blogs about more specific subjects, but my interests always seem to stray. I get interested in some other subject and start yet another new blog. This one will be a lot more general.
  2. I don't want my Myspace page to be the first thing that comes up on Google when an HR person searches for me.
  3. I am a conformist, and therfore, must have a blog just like everyone else.
  4. I have offensive, disquieting, uninformed opinions that are sure to entertain those who are smarter than me.
  5. I am a conformist and I want the DHS to put me on a watch list just like everyone else.
Things I like:

Linux: I dual installed it about 6 months ago, but just recently made the switch completely

RSS: There isn't a better way to read the internet. Hell, there isn't a better way to compute. I want everything to be fed through my reader.

Pen and Paper Games: RPG's, choose your own adventures, sudoku, board games, etc.

Writing Music: Haven't done a lot of this lately, but in a later post I'll fill you in on all my musical exploits. If you want something now you can search for "Diver" in itunes.

My Wife: She is awesome.

My Dog: Ben Franklin is stench and cuteness

Camping: Going twice this summer will not be enough

This list doesn't end, but has been abridged for this post.

Game Imbalance

As those of you who play World of Warcraft know, every Tuesday the servers go down for maintenance. Sometimes, these downtimes are accompanied by a software patch for the game. Ahh, patch day; the day of bug fixes, and most importantly, game play tweaks. The developers diligently monitor the damage per second of every character class, making sure that none of them exceed their alloted amount of influence in the game.

It isn't just World of Warcraft, but pretty much every game in the market right now. Game Balance, over the lifecycle of a game, becomes the most important and most tweaked aspect of any game. Visit any forum and you'll see hordes of players crying out about the injustices visited upon them. "Nerf Warlocks!", "Don't Nerf Pallies!" etc. etc... So, in an effort to minimize complaints, to make sure everyone is playing fairly, and most of all to ensure everyone has a good time, the developers listen to the arguments of these players and appease them on patch day.

I can understand the outcry to a certain extent. Games like this need to be balanced so that all players can have an equal influence on their surroundings and other players. If there is something in the game that is unbalanced, a player will exploit it and then they will have a disproportionate amount of influence.

I can't help but think that a game that requires this amount of tweaking is imposing an unnatural order into a system.

What if there were a game that thrived on the natural tendency for players to exploit the game for power and use it at other players expense? What if there were a game that rewarded players keeping these secrets to themselves? What if there were ways, in game, for you to steal other players knowledge and use their own tricks against them.

That sounds like a game I would love to play.

I had a small taste of game play like this in the game Shogun Wars. This game definitley had large game balance issues, but I don't think the game suffered for it, in fact, I think the game was improved by them. The best tactic was to save all your gold, improve your financial position, and stay in the bottom of the rankings avoiding notice until you can't upgrade your gold production anymore. Then, when you are producing a large amount of gold, you shoot up through the rankings by purchasing weapons and warriors.

Another exploit we used in the game was a vast database of our enemies statistics. Whenever you sent ninjas into an enemies base, you could retreive information about thier attack and defensive power. Our group made a database of this information, giving us a distinct advantage in the game. I don't think the level of our coordination was expected by the creators of the game because we ended up with about 80 of the top 100 spots in the game.

I am sure we made the game all but unbearable for others to play, especially newcomers, but our clan was big, and with so many players it was impossible to know who the spies were. many players infiltrated our clan and had access to this database. It was a very immersive experience, and one that should be more common in games.

These immersive elements came out of giant unbalanced holes in the gameplay where the players decided to make their own fun. Maybe the fun was a little tilted in our favor, but by no means should that discourage a game designer from trying to make a game where exploitation and a complete disregard of game balance can be fun for everyone...