USGS Inorganic Blind Sample Project (IBSP)
Monitoring and Evaluating the National Water Quality Laboratory's Inorganic Analytical Data Quality
The U.S. Geological Survey (USGS) collects and disseminates information about the water resources of the United States and its territories. Surface- and ground-water samples are collected and sent to USGS laboratories for chemical analyses. About 75 to 80 percent of the water-quality samples collected by the USGS are submitted to the National Water Quality Laboratory (http://nwql.usgs.gov/nwql.shtml, accessed February, 2012) in Lakewood, Colorado. The National Water Quality Laboratory (NWQL) identifies and quantifies the constituents in the water samples through analytical chemical procedures. Inherent to these procedures are random and systematic errors otherwise known as uncertainty. Although some errors can be addressed, other errors are inherent to the analytical process and cannot be eliminated. The magnitude of errors, or uncertainty, can be estimated and tracked over time.
Since 1981, the USGS has operated an independent, external, quality-assurance project called the Inorganic Blind Sample Project (IBSP). The purpose of the IBSP is to monitor and evaluate the quality of the inorganic analytical results through the use of double-blind quality-control (QC) samples. Starting in 1984, inorganic analytical results from blind samples submitted to the National Water Quality Laboratory (http://nwql.usgs.gov/nwql.shtml) have been captured and stored in the IBSP database. The Ocala Water Quality Laboratory was added to the project in 1985 and remained in the project until the laboratory closed its doors in 2004. Inorganic analytical results from blind samples submitted to both the NWQL and the Ocala laboratory are available in the IBSP database.
On-line Tools Aid in Water-Quality Interpretation
The IBSP maintains a computer program (QADATA Application) for the retrieval, assessment, and storage of blind QC sample analytical results. Currently, the QADATA Application contains more than 300,000 blind-sample results for inorganic, nutrient, and physical-property measurements dating from October 1984 to present. New analytical data are added twice-weekly to the QADATA Application. The QADATA Application provides tools to assist the National Water Quality Laboratory in detecting and correcting problems in their analytical procedures in a timely fashion. In addition, the tools can aid the users of the NWQL’s data by estimating the extent that laboratory bias and variability contribute to the overall bias and variability in their environmental data. Information is provided in three ways: 1. quality control charts (see IBSP control charts link), 2. data-quality reports posted every other month (see IBSP qadata summaries link), and 3. raw data from the IBSP database. Data retrievals can be customized to document the NWQL's analytical bias and variability relative to the time period, analytical procedures, and concentration ranges of individual water-quality projects or programs. (Contact Ted Struzeski at 303.236.1872 [firstname.lastname@example.org] for a customized data retrieval.)
What Is A Double-Blind QC Sample Used by the IBSP?
A "double-blind sample" is a QC sample submitted for analysis for which the identity of the sample as well as the concentration of the individual components within the sample is unknown to the analyst. Double-blind QC samples containing inorganic, nutrient, and physical property constituents at various concentrations are prepared and disguised as routine environmental samples. The QC samples used by the IBSP are very unique in that they are typically not synthetic reference materials; rather they are derived from snow-melt, surface-waters, or ground-water sources (Woodworth and Connor, 2003). These natural-matrix standard reference samples are used as-is (undiluted), or diluted with deionized water, or mixed in varying proportions with other standard reference samples in order to achieve a variety of concentrations within the range of concentrations that corresponds to those of typical environmental water samples.
The double-blind IBSP samples are made to appear as much like environmental samples as possible. The IBSP submits these samples in shipping coolers to the NWQL to mimic the process, as much as possible, by which actual environmental samples are submitted to the NWQL. All identifying information (except account number) is changed to that of actual customers. Bottle labels are even "soiled" go give the appearance that the bottles have been filled in the field. After the samples are logged in, they are subjected to the identical laboratory handling, processing, and analytical procedures as are the environmental samples. Once the laboratory analyzes the samples, the results are routed through the same channels as used by regular customers, at which point the IBSP personnel immediately compile and review the analytical results.
The NWQL is evaluated by how closely its analytical results approximate the target value of the blind QC samples. The target values are derived from the median concentrations listed for each constituent in the standard reference samples (http://bqs.usgs.gov/srs/) used to make the double-blind QC samples that are submitted to the NWQL. The assessment of whether an analytical result is acceptable is based on the number of f-pseudosigmas (an f-pseudosigma being a statistically robust approximation of the standard deviation) that the laboratory’s measured concentration differs from the target value. Analytical results that are within two f-pseudosigmas of the target value are considered acceptable.
Analytical Bias and Variability
Analytical errors fall into two major categories: bias and variability. Bias is systematic error that causes consistently positive or negative deviation in the results from the target value. Variability is random error that affects the ability to reproduce results. Repeated measurements of the IBSP samples over time provide estimates of both systematic bias and random variability in the laboratory’s analytical procedures.
The IBSP uses a variety of graphical and statistical tools to evaluate laboratory analyses of the blind QC samples. These tools include control charts, relative standard deviation charts, box plots of error distributions, Wilcoxon signed-rank test for bias, binomial-probability distribution test for variability, and statistical summaries.
Woodworth, M.T., and Connor, B.F., 2003, Results of the U.S. Geological Survey's Analytical Evaluation Program for Standard Reference Samples Distributed in March 2003: U.S. Geological Survey Open-File Report 03-261, 109 p.