Entropy

The average amount of information represented by a symbol in a message, is a function of the model used to produce that message and can be reduced by increasing the complexity of the model so that it better reflects the actual distribution of source symbols in the original message. Entropy is a measure of the information contained in message, it's the lower bound for compression.