Whether to pool variance or not..

Copied from,

unequal variance t-test is an underused alternative to Student's t-test and the Mann–Whitney U test | Behavioral Ecology | Oxford Academic (oup.com)

The Student's t-test performs badly when these variances are actually unequal, both in terms of Type I and Type II errors. Unequal variances are less problematic if sample sizes are similar.

we see that the Type I error rate of the unequal variance t-test never deviates far from the nominal 5% value, whereas the Type I error rate for the Student's t-test was over 3 times the nominal rate when the higher variance was associated with the smaller sample size and less than a quarter the nominal rate when the higher variance was associated with the higher sample size.

Thus, unequal variance t-test performs as well as, or better than, the Student's t-test in terms of control of both Type I and Type II error rates whenever the underlying distributions are normal.

The unequal variance t-test has no performance benefits over the Student's t-test when the underlying population variances are equal. Hence, you might consider that an effective way to conduct your analysis would be to perform an initial test for homogeneity of variance and then perform either a Student's t-test when the variances are equal or an unequal variance t-test when they are not. The problem with this flexible approach is that the combination of this preliminary test plus whichever of the subsequent tests is ultimately used controls Type I error rates less well than simply always performing an unequal variance t-test on every occasion.

this is one reason why it is generally unwise to decide whether to perform one statistical test on the basis of the outcome of another.

There are further reasons for not recommending preliminary tests of variances, ref 6&7.

It is important to remember that although the unequal variance t-test is more reliable than the Student's t-test in terms of violation of the assumption of homogeneity of variances, it is not necessarily any more reliable than the Student's t-test if the assumption of normality of the underlying populations is violated.

However, Zimmerman and Zumbo (1993) argue that the unequal variance t-test performed on ranked data performs just as well as the Mann–Whitney U test (in terms of control of Type I errors) when variances are equal and considerably better than the U test when variances are unequal. ref 8.

suggest that the unequal variance t-test can effectively replace the Mann–Whitney U test if the data are first ranked before the test is applied. 

You may also encounter the unequal variances t-test called simply the unpooled variances t-test or Satterwaite's test or the Welch–Satterthwaite test

You may also find it called as the Smith/Welch/Satterwaite test, acknowledging the work in Smith (1936).

The importance of considering whether or not to pool variances extends beyond the simple case of comparing 2 groups. Julious (2005) argues against the standard practice of using the pooled variance across all groups when performing a comparison between 2 groups from several used in an analysis of variance. 

No matter the number of groups, the decision as to whether to pool or not also needs careful consideration in the construction of randomization tests as well as the analytic tests considered here.

Reference

1. Coombs WT, Algina J, Oltman D. 

   1996
   . Univariate and multivariate omnibus hypothesis tests selected to control type I error rates when population variances are not necessarily equal. 
   Rev Educ Res
   66
   :
   137
   –79.

2. Zimmerman DW, Zumbo BN. 

   1993
   . Rank transformations and the power of the Student t-test and Welch t′-test for non-normal populations. 
   Can J Exp Psychol
   47
   :
   523
   –39.

3. Moser BK, Stevens GR, Watts CL. 

   1989
   . The two-sample t-test versus Satterwaite's approximate F test. 
   Commun Stat Theory Methodol
   18
   :
   3963
   –75.

4. Moser BK, Stevens GR. 

   1992
   . Homogeneity of variance in the two-sample means test. 
   Am Stat
   46
   :
   19
   –21.

5. Zimmerman DW. 

   2004
   . A note on preliminary tests of equality of variances. 
   Br J Math Stat Psychol
   57
   :
   173
   –81.

6. Markowski CA, Markowski EP. 

   1990
   . Conditions for the effectiveness of a preliminary test of variance. 
   Am Stat
   44
   :
   322
   –6.

7. Quinn GP, Keough MJ. 

   2002
   . Experimental design and data analysis for biologists. Cambridge: Cambridge University Press.

8. Zimmerman DW, Zumbo BN. 

   1993
   . Rank transformations and the power of the Student t-test and Welch t′-test for non-normal populations. 
   Can J Exp Psychol
   47
   :
   523
   –39.

9. Welch BL. 

   1938
   . The significance of the difference between two means when the population variances are unequal. 
   Biometrika
   29
   :
   350
   –62.

10. Welch BL. 

    1947
    . The generalisation of students problem when several different population variances are involved. 
    Biometrika
    34
    :
    23
    –35.

Satterwaite FE. 

1946

. An approximate distribution of estimates of variance components. 

Biometrics Bull

2

:

110

–4.

Smith H. 

1936

. The problem of comparing the results of two experiments with unequal errors. 

J Counc Sci Ind Res

9

:

211

–2.

Julious SA. 

2005

. Why do we use pooled variance analysis of variance. 

Pharm Stat

4

:

3

–5.

Moser BK, Stevens GR, Watts CL. 

1989

. The two-sample t-test versus Satterwaite's approximate F test. 

Commun Stat Theory Methodol

18

:

3963

–75.