Notes on Stratified Sampling

for Statistics 36-303: Sampling, Surveys, and Society

Stephen E. Fienberg Department of Statistics Carnegie Mellon University (Revised by Brian Junker, Spring 2008)

February 26, 2008

I. Why Stratify?

One of the simplest extensions to simple random sampling involves the use of stratification of the population of interest. In stratified random sampling, we divide the population into distinct subpopulations called strata, and within each stratum we select a separate random sample. There are four primary reasons for using stratification:

1. The subpopulations that form the strata are “domains” of study or interest and thus we need to get separate estimates for each stratum. For example, we may take separate samples for men and women because we are interested in making separate inferences for the subpopulations as well as comparing them.

2. The subpopulations or strata are such that different methods or sampling schemes may be suitable for the different strata. For example, if we are estimating the number of words in a textbook, we might take different samples for different sections: the front matter. the main text, the references, and the index.

3. The population is geographically diverse and thus stratification is a convenient way of organizing sampling and data collection.

4. By employing stratification in a careful fashion, we can get improved precision for our estimates of population quantities.

In these notes, we explain, both by example and by formula, the potential gains in precision due to stratification, i.e. reason 4 above. We exploit the formal results on the calculation of expectations and variances for linear combinations of random variables, from the handout distributed in class. Later in the course, we will discuss the combining of stratification with other sampling devices such as clustering.

Statistics 36-303  Spring, 2008

II. Stratification for the Estimation of Proportions

Here we are interested in estimating the proportion p of some attribute in the population (what is the proportion of yellow balls? What is the proportion of people who ride PAT buses?, etc.)

Example 1: Three Urns. Consider three urns, with the ith urn containing N_{i} balls of two types, N_{i}p_{i} yellow ones and N_{i}(1 − p_{i}) blue ones for i = 1, 2, 3. We are interested in the attribute yellow and our goal is to estimate the overall proportion of yellow ball in the three urns combined. The composition of the three urns is:

1. N_{1} = 100, p_{1} = 0.1,

2. N_{2} = 100, p_{2} = 0.2,

3. N_{3} = 100, p_{3} = 0.3.

What we would like to do is compare the precision of a stratified random sample with n_{i} = 10 balls from each of the three urns with a simple random sample of size n = ∑_{i=1}^{3} n_{i} = 30 balls from a pooled urn with composition N = ∑_{i=1}^{3} N_{i} = 300 and p = 0.2.

To compare the precision of the two different sampling schemes, we need look at the variances of the estimates of p for the two situations or the corresponding standard deviations.

IIa. Precision for Simple Random Sampling

The variance of a sample proportion, p̂, based on an SRS without replacement, of size n, from a population of size N, with population proportion of attributes equal to p, is (Groves, p. 100):

V ar(p̂) = \frac{p(1 − p)}{n − 1}(1 − f) where f = n/N.  (1)

Example 1 (cont.): For our urn with N = 300 and p = .2, the variance of a simple random sample of size n = 30 is

V ar(p̂) = \frac{.2(1 − .2)}{29} (1 − \frac{30}{300}) = 0.004966  (2)

and

s.d.(p̂) = 0.0705.  (3)

Of course if we do not know p, a good conservative strategy is to pretend p = 1/2 since that will make the variance as large as possible.

IIb. Precision for Stratified Random Sampling

Now suppose we have a population of size N divided into H strata. We take a stratified random sample with n_{h} observations in the ith stratum, which has population size N_{h} and a population proportion of attributes equal to p_{h}. Let p̂_{h} be the sample proportion in the ith stratum. Our overall sample size is n = ∑_{h} n_{h} and the “pooled” estimate of p, p̂_{strat}, weights the sample proportions from each stratum according to the population sizes from the strata, i.e.,

p̂_{strat} = ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )p̂_{h}.  (4)

Weighting the sample proportions according to the population sizes makes the stratified estimate, p̂_{strat}, unbiased, i.e., if we average over all possible such samples that are drawn without replacement, we get the overall population proportion, p:

E(p̂_{strat}) = E[ ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )p̂_{h}]  (5)

= ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )E(p̂) = ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )p_{h}
= p.

We can compute the variance for p̂_{strat} as a special case of the formula for the variance of a linear combination of independent random variables, {X_{h}}:

V ar (∑_{h=1}^{H} a_{h}X_{h}) = ∑_{h=1}^{H} a_{h}^{2}V ar(X_{h}).  (6)

Thus we have that

V ar(p̂_{strat}) = V ar( ∑_{h} (\frac{N_{h}}{∑_{k} N_{k}} )p̂_{h})  (7)

= ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )^{2}V ar(p̂_{h})

= ∑_{h=1}^{H} (\frac{N_{h}}{∑_{k} N_{k}} )^{2} \frac{p_{h}(1 − p_{h})}{n_{h} − 1}(1 − f_{h}).

Example 1 (cont.): In our example with the 3 urns, this formula for the variance of the stratified estimate simplifies because they have the same sample size, 10, and population size, 100:

V ar(p̂_{strat}) = (\frac{100}{300})^{2} \frac{1 − 10/100}{10 − 1} ∑_{h=1}^{3} p_{h}(1 − p_{h})  (8)

= (0.0111) ∑_{h=1}^{3} p_{h}(1 − p_{h})
= (0.0111)(.1 × .9 + .2 × .8 + .3 × .7)
= 0.00511

and the standard deviation of p̂_{strat},

s.d.(p̂_{strat}) = .0715,

is smaller than that for the simple random sample of equivalent overall size that we computed above. The reduction in the standard error is less than 2%. This may not seem all that substantial but it is a gain that we can achieve in a consistent fashion if we choose our sample sizes for the strata with care as we have in this example.

IIc. Gains from Proportionate Stratified Sampling

The improvement of the variance in our example of the urns is not a coincidence, but rather an illustration of an important result that explains the gain that results from stratification. If we choose sample sizes for our strata in proportion to the population sizes, i.e., we let

n_{i} ∝ N_{i},  (9)

then we can rewrite p̂_{strat} as

\overline{p̂} = ∑_{i} (\frac{n_{i}}{∑_{j} n_{j}} )p̂_{i}  (10)

and the variance of the proportionate stratified estimate as

V ar(\overline{p̂}) = ∑_{i} (\frac{n_{i}}{∑_{j} n_{j}} )^{2} \frac{p_{i}(1 − p_{i})}{n_{i}} \frac{(N − n_{i})}{(N − 1)}.  (11)

Then we can show algebraically that

V ar(\overline{p̂}) ≤ V ar(p̂),  (12)

as we saw in our example of the three urns where the sample sizes where chosen in accord with proportionate sampling. The gains that result from stratification when we sample proportional to population size are modest, however, unless the {p_{i}} for the strata are widely spread.

Example 1 (cont.): Let us consider another version of our urn problem but with two urns each with N_{i} = 150 balls. In the first urn there are no yellow balls and thus p_{1} = 0.0 while in the second urn there are 60 yellow balls (and 90 blue ones) so that p_{2} = 0.4. The pooled urn again has composition N = ∑_{i} N_{i} = 300 and p = 0.2. Now we compute the variance of \overline{p̂} for the two strata situation as

V ar(\overline{p̂}) = \frac{(15/30)^{2}}{15} \frac{(150 − 15)}{(150 − 1)} ∑_{i=1}^{2} p_{i}(1 − p)  (13)
= \frac{∑_{i=1}^{2} p_{i}(1 − p_{i})}{149}
= \frac{9}{4} \frac{(0 × 1) + (.4 × .6)}{149}
= \frac{.54}{149} = .003624  (14)

and

s.d.(\overline{p̂}) = .062.

In this situation there is a reduction of (1 - 0.602/0.694)= 13.3% in the standard deviation of our estimate. Clearly by setting up an urn with no yellow balls, and then choosing a second urn so that the overall composition of the pooled urns remains the same, we have illustrated the maximum possible gain for proportionate stratified sampling. In real life problems it is often difficult to achieve such gains and thus, when we are interested in attributes, we typically carry out stratification primarily for other reasons.

Proportionate random sampling has another feature worthy of note: By choosing a sample size for each of the strata in proportion to its population size, the method yields an equal probability of selection for each individual element of the population. As a consequence our sample estimate for p assigns equal weights to each observation. This property is often referred to as self-weighting. What this means is that \overline{p̂} is just the overall proportion of elements in the sample with the attribute of interest. Of course, simple random sampling is also self-weighting.

Example 2: North Carolina Citizen Survey. Henry (1990, p. 63) describes a series of North Carolina household surveys carried out in the late 1970s and early 1980s to examine the effectiveness of programs in human services, transportation, and economic development. The survey employed stratified sampling with separate samples being drawn from two strata:

(a) a list of 1975 North Carolina income tax returns, and (b) a list of North Carolina residents eligible for Medicaid assistance in 1975. The initial samples for the strata were selected in proportion to the sizes of the two lists. Later when the response rate from the Medicaid list was lower than that for the tax list, the proportion from list (b) was increased and the sample was no longer self-weighting.

IId. Interpreting the Gain From Stratification

One way to interpret the gain that results from proportionate stratification is to think in terms of the efficiency of the sampling design. We ask what size simple random sample would produce the same precision as that achieved by our proportionate stratified sample? Let n_{equiv} be this value. Then we could think of the savings associated with stratification in terms of the ratio n/n_{equiv}. This turns out to be the same (except for the finite population correct) as the following definition:

Efficiency of the design = \frac{V ar(\overline{p̂})}{V ar(p̂)}.  (15)

Example 1 (cont.): For our three urn-example the efficiency of stratification was \frac{.004646}{.004816} =

96.5% whereas for the extreme two-urn example the efficiency was \frac{.003624}{.004816} = 75.3%.

IIe. Stratification Versus Quotas: Beware

Many market research firms still use some version of quota sampling, in order to disguise low response rates and to “fix” to sample up. The proponents of this approach to sampling either directly or indirectly claim that you don’t have to worry about call-backs to nonresponders as long as you get the predetermined numbers of individuals from designated subpopulations. These subpopulations are then represented as strata and the quotas as the subpopulation sample sizes in stratified random sample. While this quota approach to sampling was discredited man y years ago, vestiges remain and you need to be on the lookout for signs of its use. Suspicious symptoms of quota are round numbers of respondents in designated categories and the absence of information on how the sample was drawn and what was done about nonresponse.

Example 3: Quarterly Survey of Allegheny County, Pennsylvania. The Pittsburgh Post-Gazette (February 8, 1994) reported on an a quarterly survey conducted by a market research firm of Allegheny County residents to determine attitudes toward public policy issues facing the Pittsburgh area. In the survey conducted during the three-week period from October 26 to November 13, 1993, the newspaper reported that there were 600 respondents, 300 males and 300 females, and that the poll had a plus or minus error rate of 4 percent.” The round numbers of men and women should make you suspicious, especially in the absence of any further information on the sampling method. The error rate reported in this news article sounds suspiciously like 2 times the standard deviation of some estimate of a population proportion, corresponding to an approximate 95% confidence interval. Let’s check out the calculation. Suppose we had a simple random sample of n = 600 with population proportion of attributes equal to p = .5. The population size of Allegheny county is in excess of one million and thus we can ignore the finite population correction. Thus the variance of p̂ is

V ar(p̂) = .5 × \frac{.5}{600} = .0004166

and the estimated standard deviation is .0204 or just over 2%. If we had used stratified random sampling, with a sample of size n_{i} = 300 from separate strata for men and women, and different proportions, p_{i}, then the variance of a estimated proportion would be smaller, since we would have a proportionate stratified sample. The error estimate looks like it came out of an elementary statistics textbook and reinforces our suspicion about the possible uses of quota sampling.

III. Stratification for Measurements

The same idea of using stratification works for measurements as well as for proportions, but now the gains due to a careful choice of stratification may be quite substantial. Further we can achieve even greater gains than those offered by proportionate stratified sampling.

Suppose we wish to estimate a population quantity, μ = \overline{Y}, the mean of the measurements for the entire population. For a simple random sample of size n, we use as our estimate the sample average, \overline{y}. The variance of \overline{y} is

V ar(\overline{y}) = \frac{σ^{2}}{n} \frac{(N − n)}{(N − 1)}.  (16)

IIIa. Proportionate Stratification

Now we divide the population into k subpopulations or strata of size N_{i} where ∑ N_{i} = N. For the ith stratum we estimate the stratum mean μ_{i} = \overline{Y}_{i} based on a sample of size n_{i} and the stratum sample average, \overline{y}_{i}. As before we take our stratified sample estimate to be a weighted combination of the stratum averages:

\overline{y}_{strat} = ∑_{i=1}^{k} (\frac{N_{i}}{∑_{j} N_{j}} )\overline{y}_{i}.  (17)

Just as in the case with attributes, weighting the sample means in proportion to the population sizes makes the stratified estimate, \overline{y}_{strat}, unbiased, i.e., if we average over all possible such samples that are drawn without replacement, we get the overall population proportion, μ, i.e.

E(\overline{y}_{strat}) = ∑_{i=1}^{k} (\frac{N_{i}}{∑_{j} N_{j}} )μ_{i} = μ.  (18)

The variance of this stratified estimate is

V ar(\overline{y}_{strat}) = V ar( ∑_{i=1}^{k} (\frac{N_{i}}{∑_{j} N_{j}} )\overline{y}_{i})  (19)

= ∑_{i=1}^{k} (\frac{N_{i}}{∑_{j} N_{j}} )^{2}V ar(\overline{y}_{i})

and by recalling that

V ar(\overline{y}_{i}) = \frac{σ_{i}^{2}}{n_{i}} \frac{(N_{i} − n_{i})}{(N_{i} − 1)},  (20)

we can write

V ar(\overline{y}_{strat}) = ∑_{i=1}^{k} (\frac{N_{i}}{∑_{j} N_{j}} )^{2} \frac{σ_{i}^{2}}{n_{i}} \frac{(N_{i} − n_{i})}{(N_{i} − 1)}.  (21)

In practice we typically do not know the stratum variances and so we substitute sample estimates.

Example 4: Taxpayer Incomes. Williams (1978) gives a simple illustration of gains from stratification for a small population of measurements on the income (in thousands of dollars) for a population of N = 9 taxpayers:

Taxpayer No. Actual Income ($ thousands) Reported Income
1  60  50
2  72  56
3  68  66
4  94  76
5  90  90
6  102  100
7  116  112
8  130  110
9  200  175

For the complete population we have N = 9, and for the actual income μ = 103.56 and σ^{2} = 1621.14. The variance for a simple random sample of n = 3 is

var(\overline{y}) = \frac{1621.14}{3} \frac{(9 − 3)}{(9 − 1)} = 405.285.

Now suppose we use the values of the reported income in order to stratify the population into two subpopulations consisting of the first eight observations and the ninth observation:

N_{1} = 8, μ_{1} = 91.50 and σ_{1}^{2} = 515.75; N_{2} = 1, μ_{2} = 200 and σ_{2}^{2} = 0.

We now take a sample of size n = 3, with n_{1} = 2 from stratum 1 and n_{2} = N_{2} = 1 from stratum 2. Then

V ar(\overline{y}_{strat}) = (\frac{8}{9})^{2} \frac{515.75}{2} \frac{(8 − 2)}{(8 − 1)} + 0 = 174.65.

The stratified estimate in this case has a variance which is less than half that for a simple random sample. Of course, we are rarely so luck as to be able to identify the largest element in a population and give it it’s own stratum!

Next, suppose we choose sample sizes for our strata in proportion to the population sizes, i.e., we let

n_{i} ∝ N_{i},  (22)

and we label the resulting estimate as \overline{\overline{y}}, then

V ar(\overline{\overline{y}}) = ∑_{i} (\frac{n_{i}}{∑_{j} n_{j}} )^{2} \frac{σ_{i}^{2}}{n_{i}} \frac{(N_{i} − n_{i})}{(N_{i} − 1)}.  (23)

Now some messy algebra yields that

V ar(\overline{\overline{y}}) ≤ V ar(\overline{y}).  (24)

This is the same result we had above for sampling of attributes. Proportionate stratification improves the precision of our estimate relative to simple random sampling.

Just as in the case of sampling for attributes, proportionate stratification produces an estimate \overline{\overline{y}} that assigns equal weights to the observations, i.e., if we let y_{ij} be the jth observation from the ith strata, then we can rewrite the formula for \overline{\overline{y}} as

\overline{\overline{y}} = \frac{∑_{i} ∑_{j} y_{ij}}{n} .  (25)

This is the same self-weighting property that we discussed earlier.

Example 2 (cont.): The North Carolina surveys measured, among other variables, the income of households. Thus we would expect greater precision from the stratified sample that was used than from a simple random sample. The nonresponse problem and the later adjustments to the sample discussed by Henry (1990) make the realized sample accuracy somewhat more complicated.

Example 4 (cont.): We continue with the example of the taxpayer actual incomes, but now we have two strata organized geographically: stratum 1 contains N_{1} = 6 middle income taxpayers, numbers 1, 2, 3, 4, 6 and 7, while stratum 2 contains N_{2} = 3 taxpayers, numbers 5, 8, and 9. For these strata we have

μ_{1} = 85.33 and σ_{1}^{2} = 412.79; μ_{2} = 140, and σ_{2}^{2} = 2066.66.

We again take a stratified random sample of size n = 3, but this time we sample in proportion to population size, so that n_{1} = 2 and n_{2} = 1, and

V ar(\overline{\overline{y}}) = (\frac{6}{9})^{2} \frac{412.79}{2} \frac{(6 − 2)}{(6 − 1)} + (\frac{3}{9})^{2} \frac{2066.66}{1} \frac{(3 − 1)}{(3 − 1)}
= 73.385 + 229.629 = 303.01.

This result is an improvement upon the variance for simple random sampling, but it isn’t as good when we treated the largest taxpayer as forming a separate stratum and included that individual in the sample with certainty.

Example 5: Test Scores. Henry (1990, pp. 99-106) gives a detailed example with different examples of proportionate stratification and achieves estimated efficiency gains of 40% as measured by the sample estimates of V ar(\overline{y})/V ar(\overline{\overline{y}}).

IIIb. Optimal Allocation

It turns out that we can achieve even greater gains in stratification by selecting our sample rates f_{i} = n_{i}/N_{i} in the strata proportional to the population standard deviations, σ_{i}, i.e., by choosing

n_{i} ∝ N_{i}σ_{i},  (26)

subject to the restriction that the stratum sample sizes add up to the overall sample size, i.e.,

n = ∑_{i=1}^{k} n_{i}.  (27)

This is a version of something that statisticians called optimal allocation and, if we let \overline{y}_{opt} be the estimate using these sampling rates in the strata, then some further algebra yields that

V ar(\overline{y}_{opt}) ≤ V ar(\overline{\overline{y}}) ≤ V ar(\overline{y}).  (28)

We can determine the efficiency of optimal allocation relative to proportional allocation or simple random sampling by considering the ratios of the corresponding variances. As in the case of proportionate stratification for attributes above, we can interpret these ratios approximately as determining the equivalent sample sizes required to yield the same precision. Thus the ratio V ar(\overline{\overline{y}})/V ar(\overline{y}_{opt}) gives, approximately, the relative proportional increase in sample size that would be required for proportional stratification to achieve a similar level of precision as optimal allocation. Note that using optimal allocation of our sample to strata with n_{i} ∝ N_{i}σ_{i} makes the probability of selection for the individuals in some strata higher than in others. The result is that we do not have a self-weighting sample and \overline{y}_{opt} is not an estimate which weights each observation equally. This is a price we pay for the increase in precision.

Example 4 (cont.): We continue with the example of the taxpayer actual incomes, and with the same two strata organized geographically of sizes N_{1} = 6 and N_{2} = 3 where

μ_{1} = 85.33 and σ_{1}^{2} = 412.79; μ_{2} = 140, and σ_{2}^{2} = 2066.66.

We again take a stratified random sample of size n = 3, but this time we attempt an optimal allocation so we want to choose our sample sizes such that

\frac{n_{1}}{n_{2}} = \frac{N_{1}σ_{1}}{N_{2}σ_{2}} .

Of course, we can only do this if we know the stratum variances which we do not. Thus we’ll need to use a “guess” for the ratio σ_{1}/σ_{2}. Suppose we take this to be 1/3 (the true value being .447). Then

\frac{n_{1}}{n_{2}} = \frac{6}{3} \frac{1}{3} = \frac{2}{3}.

But we can’t divide a sample of size 3 into two parts in the ratio 2 to 3 and so we will need to do the best we can allocation and . The resulting estimate is not quite , and we label it with a prime as . The variance of this approximation to the optimum allocation is

V ar(\overline{y}′_{opt}) = (\frac{6}{9})^{2} \frac{412.79}{1} \frac{(6 − 1)}{(6 − 1)} + (\frac{3}{9})^{2} \frac{2066.66}{3} \frac{(3 − 2)}{(3 − 1)}
= 183.46 + 57.407 = 240.87.

The approximate optimum allocation estimate has variance smaller than that for the proportionate stratified estimate, as the theory suggested, and we have

V ar(\overline{\overline{y}})/V ar(\overline{y}′_{opt}) = \frac{303.01}{240.87} = 1.26

corresponding to an efficiency gain of 26%. Again we have not produced an estimate with precision equal to that when we treated the largest taxpayer as forming a separate stratum and included that individual in the sample with certainty, but we have come fairly close.

You might be asking yourself why we didn’t try to achieve this extra efficiency associated with optimal allocation of our sample across strata when we were working with attributes. It turns out that the result described above does work for attributes too, but it has no practical value. This is because the population variance in the ith stratum, p_{i}(1 − p_{i}), is a function of the unknown stratum proportion, p_{i}, and thus we need to know p_{i} in order to choose the optimal stratum sample size.

IV. Disproportionate Allocation

Optimal allocation is just one illustration of why we might opt to give up the nice selfweighting properties that go with proportionate sampling. More often we choose to weigh some strata more than others because we wish to make inferences about a subpopulation and the accuracy of proportionate stratification or optimal allocation is not sufficient for our purposes.

Example 6: The National Health and Nutrition Examination Survey. The National Health and Nutrition Examination Survey (NHANES) conducted regularly by the National Center for Health Statistics (NCHS), assesses the health and nutritional status of the general population of the United States. Data are collected through interviews, physical examinations, clinical and laboratory testing, and other related procedures. NHANES provides data that allow NCHS to estimate the prevalence of specific conditions and chronic diseases and information on nutritional status and deficiencies.

Several years ago, the Congress mandated a special sample supplement to NHANES to increase the number of Hispanics in the survey and to enable accurate estimates for the Hispanic population. In forthcoming waves of NHANES, there will be disproportionate sampling for other population groups as well such as Blacks, low-income persons, pre-school children, and the elderly.

A second reason to give up the self weighting property of proportionate sampling occurs when we are less interested in estimating an overall proportion or mean and more interested in comparisons among strata. Suppose, for example, that in your group project you wish to compare attitudes about extra-curricular activities for H&SS students with those of students in all other colleges. Think of H&SS and all other colleges as two strata. Then instead of dividing your sample in proportion to the size of the colleges you might choose sample sizes for your strata to make the within stratum variances equal. Because the stratum population sizes are unequal and we are sampling without replacement our choice of stratum sample sizes will also not be equal, but they will be quite different from those in proportional allocation.

Historical Note: The formal idea of stratification appeared early in the 20th century but it was the pioneering paper by Jerzy Neyman in 1934 that popularized the approach and introduced the idea of optimal allocation, which was found also in an 1923 paper by Tschuproff. Neyman’s version of the optimal allocation incorporates differing costs for sampling in different strata.

Note on Proofs of Results: Many textbooks on sampling actually work out the messy formulas to show the gains due to proportionate sampling and to optimal allocation. Good references are Cochran (1977) or Lohr (1999).

References Cochran, W. G. (1977). Sampling Techniques. 3rd. Edition. Wiley: New York. Neyman, J. (1934). On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection (with discussion). Journal of the Royal Statistical Society, 97, 558-625. Henry, G. T. (1990). Practical Sampling. Sage: Newbury Park, CA. Lohr, S. (1999). Sampling: Design and Analysis. Duxbury: Pacific grove, CA. Williams, B. (1978). A Sampler on Sampling. Wiley: New York.