We remember from our statistical mechanics course that entropy has something to do with information about the state of a system. In 1948 Shannon made this precise, showing that entropy is the unique measure of information obeying certain simple and plausible constraints. But it seems wrong to say that the "information content" of a newspaper is related to the entropy of the letters: we are missing a notion of complexity or richness in the text. Measuring complexity is important because successful learning and generalization (and also successful physics!) requires that we find simple explanations for our observations--models with minimum complexity. Learning is something that all animals can do, and this seems to be at the heart of intelligent behavior. I will try to explain how entropy, complexity and learning are related.