Since 1981, the USGS has operated an independent, external, quality-assurance project called the Inorganic Blind Sample Project (IBSP). The purpose of the IBSP is to monitor and evaluate the quality of the inorganic analytical results through the use of double-blind quality-control (QC) samples. Starting in 1984, inorganic analytical results from blind samples submitted to the National Water Quality Laboratory (http://nwql.usgs.gov/nwql.shtml) have been captured and stored in the IBSP database. The Ocala Water Quality Laboratory was added to the project in 1985 and remained in the project until the laboratory closed its doors in 2004. Inorganic analytical results from blind samples submitted to both the NWQL and the Ocala laboratory are available in the IBSP database.
The IBSP maintains a database for the retrieval, assessment, and storage of blind QC sample analytical results. Currently, the database contains more than 320,000 blind-sample results for inorganic, nutrient, and physical-property measurements dating from October 1984 to present. New analytical data are added twice-weekly to the database. The IBSP web site, which draws directly from the database, provides tools to assist the National Water Quality Laboratory in detecting and correcting problems in their analytical procedures in a timely fashion. In addition, the tools can aid the users of the NWQL's data by estimating the extent that laboratory bias and variability contribute to the overall bias and variability in their environmental data. Information is provided in three ways: 1. a variety of quality-control charts (see "Charts" link), 2. data-quality assessment summaries posted every other month (see "Data-quality assessments" link), and 3. raw data from the IBSP database. Data retrievals can be customized to document the NWQL's analytical bias and variability relative to the time period, analytical procedures, and concentration ranges of individual water-quality projects or programs. (Contact Ted Struzeski at 303.236.1872 [firstname.lastname@example.org] for a customized data retrieval.)
A "double-blind sample" is a QC sample submitted for analysis for which the identity of the sample as well as the concentration of the individual components within the sample is unknown to the analyst. Double-blind QC samples containing inorganic, nutrient, and physical property constituents at various concentrations are prepared and disguised as routine environmental samples. The QC samples used by the IBSP are very unique in that they are typically not synthetic reference materials; rather they are derived from snow-melt, surface-waters, or ground-water sources (Woodworth and Connor, 2003). These natural-matrix standard reference samples are used as-is (undiluted), or diluted with deionized water, or mixed in varying proportions with other standard reference samples in order to achieve a variety of concentrations within the range of concentrations that corresponds to those of typical environmental water samples.
The double-blind IBSP samples are made to appear as much like environmental samples as possible. The IBSP submits these samples in shipping coolers to the NWQL to mimic the process, as much as possible, by which actual environmental samples are submitted to the NWQL. All identifying information (except account number) is changed to that of actual customers. Bottle labels are even "soiled" go give the appearance that the bottles have been filled in the field. After the samples are logged in, they are subjected to the identical laboratory handling, processing, and analytical procedures as are the environmental samples. Once the laboratory analyzes the samples, the results are routed through the same channels as used by regular customers, at which point the IBSP personnel immediately compile and review the analytical results.
The NWQL is evaluated by how closely its analytical results approximate the target value of the blind QC samples. The target values are derived from the median concentrations listed for each constituent in the standard reference samples (http://bqs.usgs.gov/srs/) used to make the double-blind QC samples that are submitted to the NWQL. The assessment of whether an analytical result is acceptable is based on the number of f-pseudosigmas (an f-pseudosigma being a statistically robust approximation of the standard deviation) that the laboratory's measured concentration differs from the target value. Analytical results that are within two f-pseudosigmas of the target value are considered acceptable.
Analytical errors fall into two major categories: bias and variability. Bias is systematic error that causes consistently positive or negative deviation in the results from the target value. Variability is random error that affects the ability to reproduce results. Repeated measurements of the IBSP samples over time provide estimates of both systematic bias and random variability in the laboratory's analytical procedures.
The IBSP uses a variety of graphical and statistical tools to evaluate laboratory analyses of the blind QC samples. These tools include control charts, relative standard deviation charts, box plots of error distributions, Wilcoxon signed-rank test for bias, binomial-probability distribution test for variability, and statistical summaries.
Woodworth, M.T., and Connor, B.F., 2003, Results of the U.S. Geological Survey's Analytical Evaluation Program for Standard Reference Samples Distributed in March 2003: U.S. Geological Survey Open-File Report 03-261, 109 p.