Point estimation refers to the process of estimating a parameter from a probability distribution, based on observed data from the distribution. It is one of the core topics in mathematical statistics. In this chapter, we will explore the most common methods of point estimation: the method of moments, the method of maximum likelihood, and Bayes' estimators. We also study important properties of estimators, including sufficiency and completeness, and the basic question of whether an estimator is the best possible one.
Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.—John Tukey, Annals of Mathematical Statistics, 33 (1962).