This discussion is taken from Peterjohn et al. (1997). Please refer to the original paper for more details.

Choosing the best procedure for analysis of BBS data is difficult. Comparative analyses provide limited insights into the problem because they can only be used to evaluate differences among procedures, without providing clear evidence of which procedure is superior. However, we can use comparative analyses to draw some inferences about the relative merit of alternative procedures. First, if one procedure tends to consistently provide results that differ from the other, one of the procedures is likely to be biased. Also, large differences in estimated trends between procedures for individual species might indicate a biased result for that species. Second, if one procedure provides results with smaller variances, it is more efficient than the other procedure (provided that estimates [and variances] are nearly unbiased).

To document how the James et al. (1996) analysis procedure compared with the Estimating Equation results, we used their procedure to estimate survey-wide trends for 347 species for the interval 1966 - 1995. We used the methods outlined in James et al. (1996), with a few exceptions, as noted elsewhere.

Of the 347 species that were seen on >14 survey routes, we tested for differences among trends, sample sizes, and variances of estimates by calculating the mean differences across species between the 2 procedures and testing for statistical significance using paired t-tests. The NSRR procedure omits routes that do not meet minimum data criteria of at least 7 years of data, with at least 1 observation in each interval 66- 72, 73-75, 76-80, 81-83, and 84-92; this caused the sample sizes for NSRR estimates to be on average 396 routes less than those of the EE estimates. The NSRR trend estimate was 0.126 %/year larger on average than the EE estimate, which is not a significant difference (t = -0.623, P = 0.53). NSRR estimates were also significantly less efficient than EE estimates (for species with samples from more than 14 routes, mean difference in variance = -18.8, N =344, t = -2.67, P = 0.0078; for species with greater than 50 routes, mean difference = -2.63, N = 240, t = -3.52, P =0.0005).

We identified only 33 species for which estimates from the 2 methods appeared to produce a significantly different result. Note that we use the word "appeared," here- see Peterjohn et al. (1997) for the details of this comparison. This was about the number that would be expected to differ by chance; we concluded that there are not a disproportionate number of species for which the trends differ between procedures. Examination of species with significant differences indicated that most of the better-sampled species (i.e. those with larger sample sizes, and larger relative abundance) tended to differ in magnitude (not direction) of trend. Several species (e.g., Northern Cardinal) had very small estimated variances from the NSRR procedure; hence relatively minor differences in magnitude led to significant differences in trend estimates.

We concluded from this analysis that the NSRR-based estimation procedure has no obvious benefits over the EE estimation procedure. It appears that Estimating Equations produce more precise results, based on larger sample sizes, and the use of the "linear" estimation procedure does not lead to biased estimates relative to those produced by NSRR.

James, F.C., C.E. McCulloch, and D.A. Wiedenfeld. 1996. New approaches to the analysis of population trends in land birds. Ecology 77:13-27.

Peterjohn, B. G., J. R. Sauer, and W. A. Link. 1997. The 1994 and 1995 summary of the North American Breeding Bird Survey. Bird Populations 3:48-66.

Sauer, J.R., B.G. Peterjohn, and W.A. Link. 1994. Observer differences in the North American Breeding Bird Survey. Auk 111: 50-62.