Skip to main content

Information Theory

This is a very highly theoretical problem. What do you call an "Information"? To know that you have been informed. What does that mean? Analyst are always tyring to find the information from a mathematical functions and applying a probabilistic approach, from the trend or clusters of data.

But, what is information and how can you measure the value of the information. Just recently I had posted a article on modeling where, I talked about the Information Value. I might have skipped by saying that it has a mathematically rigorous process and needs in depth understanding of Information Theory. Well, let me try to explain it in as many few words as possible.

Information is something you don't know. It is something which you don't expect to happen. For example your future- so if someone tells your future, its an information. Is that confusing??

Actually, its quite simple. If someone tells you something you already know, then its not an information, cause it doesn't have any value, as you already know it. 

Therefore, even in a collection of data and in the correlation process, we try to figure out the best possible sets of variables that have the capability to predict the outcome variables. 


Lets take another example:

Lets say, you know that your schedule for train is at 12pm at night. Cause its in the ticket.

Now if someone says, that your departure time is 12pm....hmmm is that an information.

But, if someone comes to you and says, your train has been rescheduled due to the delay. Then that is an information- an information, that is useful. Anything not useful is not an information. (From what I have explained)

Mathematically, Information is inversely proportional to the probability of the occurrence of an event.

If p(E) is zero than it contains INFINITE information.
if p(E) is 1, which is the largest, then it contains ZERO information.


I hope that was simple enough.

Remember that the information that I am talking about is not the collection of a data. It is something that has value in terms of its existence.




Comments

Popular posts from this blog

Good Ad Versus Bad Ad

At InRev , for past few days we have been working on our new project. The project is about collaborative blogging, from the blogger around the world. As Bhupendra and I have been blogging for long time, with his immense experience, our team has been working to revamp the globalthoughtz and create a new blogging experience. In this site, we are planning to add a section where people can add their advertisement. We thought lets start by our own :), with already so many products on the line, it was obvious. So there goes an effort to create a small 128 px by 128 px logo. If you are a designer, you know that designing is a very time consuming process. For hours, you just test around with colors. Remember there are 256 X 256 X 256 combination of colors!! Its hard job selecting one. Lets start with F-Cube : In short F-cube is a Free Economic Reporting Site. Let me exhibit some Ad made for it and its pros and cons. (Please don't click on the image. Wait till the end. Thank You) OK, h...

Bruce Lee

Yes. Bruce Lee . Remember all the punches and kicks in his all time hits movies. But why am I talking about him. Am I going to talk about his movies? No, I am not. I am going to say about his philosophies which have been so many times projected in his fighting styles. By the way, he also wrote a book. Many may not know but he had a unique way of thinking about the life and his art of fighting. Let me put forwards his thinking, in my own words.   He believed in efficiency and rejected any amount of redundancy that affected the performance. He believed that there is no one style which is correct. Everyone has their own unique way and must harness those skills rather waste their time in doing something that is not efficient at all.  There are many fighting styles carrying their own norms and values. Every style starts by assuming something. But slowly those assumption are taken as a Gospel truth, and people make them into rules which cannot be broken. But, without understanding the key me...

Being an Artist and Being an Entrepreneur is almost similar

Most people flock to big cities to pursue their dreams and struggle for years. Some people succeed but many still find themselves waiting for the opportunities. They consider that opportunity is the biggest thing that can actually leverage their talent or skill. But I believe that it is not always so. There are couple of examples that proves that opportunity along is not everything. Moreover in today’s media hyped world, opportunities are everywhere from a reality TV shows to your own youtube channel, twitter account or facebook profile. If you look around in youtube, there are lots of artists doing great with their skills and scaling them to offline. This gives a lot of encouragement to all those people who want to pursue their passion. But there is a reality beyond pursuing your dreams that everyone considers once in a while. It’s how far you can take yourself with what you have. Most people get stuck on it and lose their passion in mid way, and some people simply cannot go beyon...