The Role of Statistical Independence in Finding Optimal Estimators

Naturally in choosing a point estimator we are interested in choosing an estimator that minimizes the risk function for all parameter space values. In practice, this is not possible due to the large number of estimators. One way to solve this problem (find optimal estimators) is to limit the range of estimators and find the best estimator in the finite range. This limitation leads us to two types of optimal estimators, namely the best equivariant estimator and the minimum risk unbiased estimator, respectively, in terms of limiting ourselves to the class of equivariant or unbiased estimators. In this paper, the role of independence in simplifying the calculation of these estimators is examined. We also deal with the stochastic independence of an invariant function and its comparison with the Basu’s theorem. To find the optimal estimators, the class of estimators can be limited. This limitation can be applied to the class of equivariant or unbiased estimators, which leads to two types of optimal estimators, namely the best equivariant estimator and the minimum risk unbiased estimator, respectively. For this purpose, in addition to the Rao-BlackwellLehmann-Scheffé theorem (Casella and Berger, 2001), two other methods have been proposed by Sathe and Varde (1969) and Eaton and Morris (1970) which can be useful to achieve this goal. In these two methods, by limiting the class of equivariant or unbiased estimators, the estimator with the minimum risk is considered as the optimal estimator. Independence can play a key role in making it easier to calculate the risk function of the best equivariant estimator and the minimum risk unbiased estimator. In fact, the role of independence is to eliminate the conditional probability in calculating the risk function of the best equivariant estimator and the minimum risk unbiased estimator, which in most cases results from Basu’s theorem and the independence of ancillary statistic from complete sufficient statistics. Similar to Basu’s theorem, it can be shown that in certain circumstances an invariant statistic and equivariant function are independent of each other, which can play a role in eliminating the conditional probability by the independence of the maximum invariant statistic from the equivariant sufficient statistic. It is noteworthy that in this case, the completeness assumption of Basu’s theorem has been replaced by equivariance assumption and the ancillarity assumption of Basu’s theorem has been replaced by invariance.

We first introduce the definitions that are needed. In the second part, by limiting the class of equivariant estimators, we create a type of optimal estimator called the best equivariant estimator and show that in groups that act as transitive on the parameter space, an invariant function is independent of the equivariant sufficient statistic. In the third part, by limiting the class of unbiased estimators, we make a type of optimal estimator called minimum risk unbiased estimator, which in a special case where the square error loss function is the same as the minimum variance unbiased estimator, which in the fourth part, in addition to Rao-Blackwell-Lehmann- Scheffé theorem (Casella and Berger, 2001), introduce two other methods proposed by Sathe and Varde (1969) and Eaton and Morris (1970) which, with the help of independence, provide a simpler method for finding a minimum variance unbiased estimator.

The following conclusions were drawn from this research.  In order to find the optimal estimators by limiting the class of estimators to the class of equivariant or unbiased estimators, the independence of complete sufficient statistics from ancillary statistics and applying Basu’s theorem can be a way to simplify calculations.  In some statistical problems with the transitive transformation group, the equivariant function can be used instead of the complete sufficient statistic. In this case, instead of using the Basu’s theorem, the independence of an invariant function and the equivariant sufficient statistic can be inferred. Hence, the assumption of completeness for establishing the Basu’s theorem is replaced by the equivariance and having a transitive transformation group.  Having a transitive group, any invariant function is also ancillary, but the incompleteness of a sufficient statistic can also result in its independence from an invariant statistic. If the group of transitive transformations and complete sufficient statistic is also equivariant, the case of Basu’s theorem concludes this view. The opposite is not always true and can be corrected in such a way that if a complete statistic is also equivariant with a transitive group, it is not necessary that each ancillary statistic be independent of it. Rather, it is possible to find an ancillary statistic that is also invariant, which is independent of the given equivariant complete sufficient statistic. By finding the condition that an ancillary statistic is also invariant, these results can be extended, in which case Basu’s theorem is the result. Of course, this open problem needs further research and it is hoped that researchers will be diligent in generalizing it.