Regional Credibility Measures

Although the BBS provides a huge amount of information about regional population change for many species, there are a variety of possible problems with estimates of population change from BBS data. Small sample sizes, low relative abundances on survey routes, imprecise trends, and missing data all can compromise BBS results. In the complicated model fitting procedures presently used for the BBS, it is possible that convergence failures in regions with these deficiencies can produce results that are inaccurate as well as imprecise. Often, users do not take these problems into account when viewing BBS results, and use the results inappropriately.

To provide some guidance to interpretation of BBS data, we have implemented a series of checks for some attributes that we view as cause for caution in interpretation of BBS results. We categorize BBS data in 3 credibility categories:

This category reflects data with an important deficiency. In particular:

• 1. The regional abundance is less than 0.1 birds/route (very low abundance),
• 2. The sample is based on less than 5 routes for the long term (very small samples), or
• 3. The results are so imprecise that a 5%/year change (as indicated by the half-width of the credible intervals) would not be detected over the long-term (very imprecise).
Users should be aware that a variety of circumstances may lead to imprecise results. For example, imprecise results are sometimes a consequence of a failure of the models to converge in those local areas, even though the model performs adequately in larger regions.

• 1. The regional abundance is less than 1.0 birds/route (low abundance),
• 2. The sample is based on less than 14 routes for the long term (small sample size), or
• 3. The results are so imprecise that a 3%/year change (as indicated by the half-width of the credible intervals) would not be detected over the long-term (quite imprecise), or

Note:

• 1. Due to changes in the way N of samples (in this analysis, it is defined as the N of routes on which the species occurred), relative abundance (taken directly from the hierarchical model results), and the precision (half-width of the credible intervals), these categories are slightly different than those used in earlier analyses. We will refine these groupings based on user comments and our evaluation of the limitations of the hierarchical model analysis.
• 2. Even data falling in the category may not provide valid results. There are many factors that can influence the validity and use of the information, and any analysis of BBS data should carefully consider the possible problems with the data. As noted above, judging whether technical issues associated with model convergence are leading to imprecise results can be difficult in analyses based on many strata, but these categories help users to screen for suspect results.
• 3. We are occasionally asked to identify which deficiency is causing the flag. However, the point of the codes is to provide a quick and simple set of cautions to users, and we are resisting the notion of setting up a complicated series of codes. To determine why the code exists, look at the results. All of these deficiencies (abundances, precisions, etc) will be evident from the results we present.