General / Off-Topic A question about information (theory)

1589459830537.png
A

1589459846708.png
B

If we have two image files with 2x2 pixels (4 pixels total) like the ones above, and the pixels can either be black or white (0 or 1), we need 4 bits to create each file.

Both images above contain the same amount of bits, and they contain the same amount of black and white pixels, yet they are obviously not the same.

If we read the files top-left to bottom right, we could write them as:

A: 0110
B: 0011

I do see that the sequence is "important", but what describes the sequence except for the information in the files (the 4 bits)?

Is there information hidden in the sequence of the bits? In that case what is that information called, and can it be measured?

Any explanation highly appreciated. Thanks in advance.
 
There's a whole thing called "bit sequence independence" in communications, which seeks to eliminate errors from reading the information wrongly. AFAIK, it only works for one dimensional strings of code, so you cannot transmit a matrix like in the example.

The general agreement on how to read the sequence should be covered by a protocol? Don't know much about this, sorry. Interesting though.
 
Thanks :)

Normally we say that physical information is needed to describe something. In the the example above the information is the "something" itself, and therefore it seems to me that there is more than the four bits in the files. There is also some sort of emerging property that we could call the pattern, but that should already be described by the information in the file. Also, if you transfer information, you must incorporate time. Of course time is a part of the Universe (everything) as is information, and of course they relate, but it seems strange to me, that you need time to thingamajig the information.

(lots of messed up nomenclature, and I find it really hard to describe in words, why this is wierd, but I have a gut feeling that it is.)
 

Ian Phillips

Volunteer Moderator
What you have there is a sequence of information: 0011.
That in itself doesn't mean anything unless it is interpreted.

So your example interprets a 1 as a white pixel. It could as easily be a black pixel - or a red pixel. It depends on the context into which the information is placed.

I look at raw data every day at work. EG 009356722
Depending on the definitions being used, that could be an account number, an insurance number, a personal ID number. There is actually no way of knowing what it is, without also knowing the definition, or context, that applies to that sequence of numbers.
 
Thanks for the answer.

My example is a thought experiment, and I deliberately made it simple. I know of no 4 pixel 1 bit image files in real life. I agree that my question doesn't have "meaning" without some sort of context, so let me try to explain it better.

Years ago I started experimenting with astrophotography. That led me to experimenting with what data (information) really is. I started out by finding as many different amateur astronomer images of the Andromeda galaxy as I could find, and then I aligned all the images, added each pixel of all the images and divided the value by the number of images I used. This method is well known, called Shift-Add og Stacking, but it had never been done with hundreds or even thousands of images. The result was quite mindblowing and time consuming.

During the home isolation these days, I decided to try and make an image again. This time however, I wanted to be more analytic in my approach. I went for the galaxy called NGC 4565, and you might not think so, but crawling the internet, I managed to collect roughly 600 different images of the galaxy. The goal was also to see how "deep" I could get the image, being how distant objects could I see. I'm not finished, but I have some results. It seems that looking at my current result, the distant galaxies in the image is roughly 10-15 billion lightyears away from Earth.

Deep_and_noise_Preview02.jpg

(click to enlarge)

I've worked a lot with Entropy and Information, and since the task of making images like the one above is a question of separating signal (stars galaxies etc) from the noise, I have been looking at the way you normally do that (there are several ways), to see if I could come up with something better. That ended up with my question described in the OP.

The question is more general than my actual task of making an image of the Universe. Information and Entropy are closely related, to the point were some even argue that they are the same thingy:

Source: https://www.youtube.com/watch?v=sMb00lz-IfE


I was thinking more about all this yesterday, and of course it doesn't make much practical sense without an observer, perception and a lot of other factors, but that only makes my lack of understanding it more complex and thereby difficult.

Instead I've tried to look the other way, towards the basic axioms of Information Theory. Instead of transferring the information, I decided to look at information that isn't being used, like the pixels in an image file. They just exist in the thought experiment above, but they are still "stable" information (it takes energy to flip a bit). The information in an image has some sort of pattern that we can decode using eyes and brain, just like a spoken sentence carries information that we can decode by understanding the words and their meaning.

When you speak you transfer bits from your own brain to the brain of the person receiving the message. Some information get lost during the transfer, like when you use words the receiver doesn't understand. Basically Claude Shannon discovered much of this back in 1948. Likewise in the image of the galaxy, there is information in what we perceive as noise. Once the signal to noise ratio get close to 1 it becomes difficult to distinguish signal from noise, but the information is still there, and by adding and dividing all the images, thereby increasing the SNR, I'm able to extract more signal, but it is to a certain degree a mystery why this works.

I find information and Entropy very interesting in many cases, and even though they connect the dots between Quantum Mechanics, Thermodynamics, Biology, Computing, and much more (with remarkable resemblance though still different), we still have a very limited understanding of what Information and Entropy really are.

So: I could arrange bits in a file so that the file carries information about how a galaxy "looks", or I could rearrange the exact same amount of information, even with what seems to be the same amount of Information Entropy, so that the image would be perceived as a picture of Cleopatra. I still don't understand why. It's the same amount of information, but it's different patterns, message, information or whatever it is called. That to me seems to indicate that there is more information in the files in the OP than just the four bits.

Cleopatra.jpg
galaxy.jpg

Seen from outside those files could be reduced to the number of bits they contain, but those bits alone doesn't show the pattern unless they are arranged in a very strict manner.
 

Ian Phillips

Volunteer Moderator
Yes - that's basically the point I was making. Interpretation is what gives information it's meaning, but that is then dependent on what the 'filter' does to that information.

INFO ...... |filter1|
1589545964459.png


INFO ...... |filter2|
1589546014985.png


To a Thargoid, filter1 would probably convey some meaningful information, filter2 not so much!

As for Information and Entropy:
I have wondered for some time a possible link between Entropy and Time.

Both are non-reversible. =======>
If (as I have picked up from somewhere) Information cannot be destroyed, then it also increases as the Universe ages, just as Entropy and Time.
 
I have also pondered the relation between time and entropy. As one of my favorite physicists, Sean Carroll says: Things will be changed about what we believe we know, but if you come up with any hypothesis that does not correspond with the 2nd Law (of Thermodynamics), then you can be sure that it's wrong. Just like Pytagoras said that even one day in the distant future, when the Universe doesn't exist anymore, the will still be right triangles. He really loved those triangles.

Regarding information, I'm not specifically thinking about information through perception. There is, to the best of our knowledge, information all over the Universe, even in the parts of Universe we literally have no way of changing or observing, beyond the event horizon of the Observable Universe. My question is more general and includes even information way out there.

Our perception of the two images is just an example of how information seems to work. It still doesn't explain why the order of the information matters, if it even does. Information has other roles in the Universe than just showing us how to recognize Cleopatra. It also tells our cells how to produce vital proteins, and it seems to govern everything we know. Even the laws of nature couldn't exist without information, just like there could be no change without time. And yes, change might also only be something we perceive.

I might very likely just be confused about the whole concept of information. I think that information is woven into reality just like time, entropy and the particles from the standard model. Information, as far as I know, cannot be destroyed, but it can be changed. That takes a certain minimum amount of energy called the Landauer limit.
 
Last edited:

Ian Phillips

Volunteer Moderator
Sorry about the lack of response, I've been spending about half of my weekend in bed, recovering from some sort of infection.

I'm following up several lines of research that have been made public, including some pf the links you have provided, and to be honest it's pretty damned involved!

Probably too far beyond my abilities to Grok at the moment.

Taking the example of water molecules from the video, where information held in one molecule, in the form of orientation, total energy and vector. When a collision occurs that information tells the second water molecule how to react. IE, whether it should now spin, what direction it should now move in, what total energy is should now have. But, those things are determined by well known laws of physics and both molecules affect the other, so it is already a much more complex interaction than a simple passing of information from one thing to another. there is a level of 'interpretation' at play, which are the physical laws.

So I keep coming back to a basic idea that information and interpretation co-exist, and that they are interdependant.
 
Back
Top Bottom