Information Theory and Memory.

You can probably file this one under 'crackpot theories' but lately I've been thinking about ideas, information, and how they spread. I am sure anyone who reads enough Neal Stephenson (link) will probably come to the same conclusions as I have, but writing it all down in an open, public, and embarrassing forum can only help deepen my understanding. Either that or I'll gain enough ridicule to stop thinking about this stuff altogether. Either way I win.

Information theory is a broad and nebulous discipline that straddles math, physics, language, and enough other subjects to be patently interesting to anyone interested in punishing themselves by reading thick books. I have yet to delve much into it beyond skimming the surface but there is one example that I read that sums up a nice little chunk of it. Say you flip a coin one thousand times, and you want to let me know, in order, whether each coin flip was heads or tails. This should be easy enough. You can just call me up on the phone, and read me your list of one thousand results.

Think about how much information that is though. That is 'Heads' or 'Tails' one thousand times. That is 5000 characters worth of information. Heads and Tails each has 5 characters. So lets represent each coin flip with a number; 0 for heads, 1 for tails. Now that is more like it. We've got 1000 coin flips and 1000 'bits' of information.

Now lets pretend your coin has some nasty crap stuck to one side of it, and now your coin almost always lands heads up. In fact, you've done the experiments, and you know that this coin will land tails up only once for every 1000 flips. I call you up and ask for the results of your latest experiment. Well, you could just go right ahead and give me a stream of 1's and 0's just like you were doing when you didn't get some nastiness all over your shiny new coin. I would have all of the information of your coin flip, but since you and I both know that there is only one tails in the whole sequence, wouldn't it be easier to just tell me where it was? You could tell me that the tails showed up in space number 33, and I wouldn't have to listen to you list off your whole big stream of numbers.

What is going on here, is you have used your predetermined knowledge of how your number set will turn out to compress your information. Before, I was listening to you spout off a list of ones and zeros for 15 minutes; now our phone conversation is about 10 seconds. You have gone from giving me 1000 bits of information to giving me the same amount of information, but in only 6 bits (33 in binary is '10 0001', the location of the one). This is the fundamental basic of information theory and the idea behind all kinds of lossless data compression that you use on your computer every day.

We know how to compress data on a computer. This is a technology that has been around for awhile now, and they have applied it to almost anything that can be stored on a computer. The compression is based on the symbols that the computer uses. There are only 2 symbols, used to store the information and the computer is made to recognize and operate on vast amounts of these symbols; but what about other storage medium. What about the human brain?

This is where the crackpot section starts and I start to speculate some stuff. These aren't completely uninformed opinions. There is a cursory understanding of the involved sciences going on, but by no means is it a deep or meaningful understanding. If you find fault with anything, please publically flog me, but understand I am speculating here.

The brain recognizes a whole different set of symbols than a computer. Through evolution, mankind has had their brains hard wired to recognize symbols according to their importance in survival and passing on our genes. We are good at remembering smells, other human faces, unique visual stimuli, subtle nuances in sounds. Our brains are wired up to the five senses, and it seems as if we store this information pretty efficiently. Sure, we are capable of using other symbols such as writing, or pictographs, or even the binary code of computers, but those are symbols that are used to abstract our senses into different symbology. When you read a word, or a series of words, it creates a picture in your mind based on your experiences. You understand the words because you can relate them to unique experiences you have had via your senses.

In the coin example, a large amount of information was stored with a very small amount of information. This was possible because the occurrence of the number 1 was so extremely rare in our set, that we could convey it in a simplified form. The symbols 1 and 0 allowed us to convey the message differently with the same symbols. We used 1 and 0 to represent a larger number rather than just heads or tails. We embedded a different symbol within our limited symbol system.

Is it possible that the brain does, or is capable of doing the same thing? We remember unique sensory experiences very well. Is the rarity of such an occurrence more easily compressed in our memories because of the same principals of information theory?

My memories are mostly of myself, other people, and food. In that order. All of the math, history, science, etc. that I have learned in school pales in comparison to the vast amount of data I have stored in my head about other people and my own sensory experiences. It is harder to remember something that didn't directly happen to you. You have to solidify the concepts out of an abstracted data set (read the words, translate them into ideas). That extra step can be a hindrance to learning because words, letters, and numerals are not the symbols our brains natively use. They use direct sensory experiences. Is it possible that sensory experiences can compress themselves in our brain because that is the symbology our brain uses natively?

(find the link to the guy who can do crazy math in his head). My speculation is that this guy has the ability to encode numbers in his head as a native data type (ie sensory input) in much the same way we would remember someone's face. (find the link to the piano synesthesia girl) I also think that because this girl can encode music in her head as more than one sensory experience it makes learning music exponentially easier for her.

What if we were to take this concept into effect every time we wanted to learn something in order to compress the knowledge into our heads. Next time you are studying for a history test, try eating a new dish you have never had before or listening to a new kind of music you are unfamiliar with that corresponds with the subject matter of the test. If you are studying for a French language exam, try cooking French foods and listening to French composers. Try studying in a new and unfamiliar location. It just may be that the unique experience you give yourself may help you compress all that dry information as a unique and rare experience.

I haven't tested my theory yet, but I'll be sure to give you a full report on how it works when I do.