The selection of metrics for ecological programs naturally flows from the Program's goals and objectives. Once identified through the goals and objectives process, a set of potential metrics may be further refined through consideration of logistical and statistical considerations. One statistical consideration is the power to detect effects of management-important magnitudes.

Statistical power represents the probability of detecting an effect—given a number of design and statistical details, and under the assumption that the effect is truly present. We accept that we may not always detect what is happening in the real world—but we would like our probability of detecting important effects to be as high as possible. We become more likely to detect an effect as (1) the magnitude of the effect increases, (2) variances decrease, (3) the number of samples increases, and (4) our level of acceptance of false positive outcomes increases. ("False positives" represent occasions when we decide something has changed but, in fact, the apparent change represented merely background variation.)

We caution that power is a planning tool and is not generally considered useful retrospectively (i.e., once data have been collected; Hoenig and Heisey 2001). For this reason, this page addresses power to detect future effects only. Interest in past performance of the designs associated with LTRMP metrics may be addressed by examining confidence intervals: means or trends contained within a given confidence interval appear reasonable while those outside the confidence interval do not (given the data and the selected Type I error rate). For example, a trend estimate with 90% confidence interval of 0.05 (0.01, 0.09) units suggests evidence against negative or negligible (<0.01 units) trends but that trends of, for example, 0.02 or 0.07 are reasonable. This approach may best be coupled with an assignment of ecologically negligible effects. If, for the present example, ecologically negligible effects had been defined as less than 0.01 units in magnitude, we would see evidence for ecologically significant effects (because the confidence interval excluded effects smaller than 0.01 units).

A previous study examined the power to detect annual changes in a number of LTRMP metrics. In the present investigation, we examined the power to detect long-term temporal trends in selected metrics. Given the typically relatively low cost of false positives for the LTRMP, we assumed a false positive or Type I error rate of 20%. For discussion purposes, we have assumed adequate power to be defined as being greater than or equal to 80%. Power to detect trends of 1, 3, 5, 10, and 20% per year were assessed and for periods of 3 through 20 years.

The following plots represent a sampling of the kind of power plots that can be created. For example, under the present sampling program, power to detect a 5% per year change in mayfly (Ephemeridae) counts ("relative abundance") in Pool 13 is expected to reach 80% after approximately 14 years (Figure 1b). This period decreases slightly to 13 years when sample size within sampling events is doubled and increases to 16 years when sample sizes are halved (Figures 1a, c). These small changes demonstrate an important issue: Given some minimal sample size per year, the sample size that really matters is the number of years sampled. This is because temporal trends are estimated across annual means—and annual means, after adjustment for trend, are often not constant. Power to detect trends of a given magnitude was higher in Pools 4 and 13 than in Pool 26 (Figures 1b, 2).

Reallocating the present macroinvertebrate sampling effort in Pool 13 entirely to a stratum with relatively low sampling variance (the impounded stratum) produces only minor effects on power to detect changes in mayfly relative abundance (Figure 1d). This is probably because the variance in the impounded stratum is similar to that in the backwater stratum and because the remaining strata (channel strata) represent only 10% of the Pool 13 sample area (for macroinvertebrates).

Comments on power to detect trends for components are provided under the headings for those components.

a

b

c

d

a

b

Figure 2. Estimated power to detect temporal trends in mayfly (Ephemeridae) relative abundance under the present sampling intensity in (a) Pools 4 and (b) 26.

Bluegill (Lepomis macrochirus) Catch-per-unit Effort

Power to detect a 5% per year trend in bluegill catch-per-unit effort (CPUE) in Pool 4 is estimated to reach 80% after approximately 17 years; doubling and halving the present sample size will change this period to roughly 16 and 18 years, respectively (Figure 3). This relatively small influence of changes in sample size is explained by the relatively high proportion of annual variance attributable to real changes in bluegill CPUE means in Pool 4 (82%) and the consequently small proportion of interannual variation attributable to variation within years (18%). These power plots are adjusted for moderate temporal correlation (= 0.47) among annual bluegill CPUE means. Power estimated under a naive assumption of temporal independence, of course, appears higher (compare Figures 3b, d). Power to detect trends in bluegill CPUE under the present sampling design is estimated to be highest in Pools 4 and 8 and La Grange Pool, lowest in Open River reach, and intermediate in Pools 13 and 26 (Figures 3b, 4). The proportion of interannual variation attributed to sampling error is highest in La Grange Pool. Consequently, power to detect a given trend in bluegill CPUE means will probably be most sensitive to changes in sampling effort in La Grange Pool.

a

b

c

d

a

b

c

d

e

Water Quality

Suspended Solids

Changes in sample size within sampling events have little influence on power to detect trends in log-transformed suspended solids levels in spring in backwaters of Pool 13 (Figure 5). This is because the majority of the variation among annual means derives from year-to-year rather than within-year or sampling variance. Of course, changing sample sizes within years affects within-year ("sampling") but not year-to-year variance. The sensitivity of estimated power was even lower for backwaters in spring and for the main channel region in both spring and summer. In these instances, >95% of the observed variation among annual means was estimated to derive from year-to-year variance; number of years to attain 80% power to detect a 3% change per year was 20 or more years. Plots for Pool 26 data appeared similar to those derived from Pool 13 data.

a

b

c

Other Water Quality Constituents

Power plots for chlorophyll and nitrate levels in backwaters and main channels in spring and summer in Pools 13 and 26 appeared similar to those reported above for suspended solids.

Methods

Estimated annual means from the LTRMP were assumed to exhibit variance at both within-year and annual scales. The within-year variance divided by the number of samples in a given year represents the sampling variance of the given annual mean, while annual variances were assumed constant regardless of within-year sample size. For the water quality outcomes, the within-year and annual variances of the means were both estimated from raw data (suspended solids, chlorophyll, and nitrate values were log-transformed prior to analysis). For counts (fish, macroinvertebrates), power was estimated from poolwide means rather than from raw data (for an improved approach, see Gray and Burlew 2006). Count means from a given pool, after log transformation and possibly after adjustment for temporal

These results assume no multiyear trends in detection probabilities (i.e., that, on average, LTRMP field personnel have not and will

Reference

Gray B.R. and M.M. Burlew. 2007. Estimating trend precision and power to detect trends across grouped count data. Ecology 88: 2364-2372.

Hoenig J.M. and D.M. Heisey. 2001. The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician 55:19-24.

Disclaimer: This information is provided to help better inform ongoing LTRMP partner discussions. The power plots are generated from small sample sizes; all plots should be treated as approximate.

Contact: Questions or comments may be directed to Brian Gray, LTRMP statistician, Upper Midwest Environmental Sciences Center, La Crosse, Wisconsin, at brgray@usgs.gov.