This is a short book by Alan Stuart that describes very simply the concepts of random sampling needed for statistical analysis with unbiased estimators (such as the sample mean).
It defines the Central Paradox of Sampling:
Once a sample has been obtained it is impossible to tell by inspection of the sample whether it has been obtained by a simple random sampling mechanism or not.
