202409231731 Status: #idea Tags: #compression #learning_theory #philosophy_of_science # Science is lossless compression Throughout history, we have compiled scientific data on a wide variety of phenomena. Typically, as this data increases in quality and size, we find patterns that cannot be explained by our current theories (or for which we had no theory at all). In this case, we cannot summarize the data without losing significant information - we can do things like take means of variables, produce plots, etc. that describe the data, but all of these coarse-grain what we've collected and lose some details on the underlying processes. On the other hand, when we learn a firm theory to explain data, we can express infinite datapoints in an incredibly compact expression. For example, the equation $\vec{F} = m \vec{a}$ captures the information for infinite measurements of mass, acceleration, and force using a very small number of bits (just the amount required to encode that string). Hence, the act of formulating and validating a scientific theory is the act of lossless compression of a large number of data points. [[Learning is compression and compression is connection]] --- # References