The U.S. Environmental Protection Agency (EPA) recently proposednew guidelines forcarcinogen risk assessment (1). The proposed guidelines areintended to replace the existingguidelines that were issued in 1986 (2). The 1986 guidelinesrequire, among other things, thatepidemiologic studies be statistically significant where they areused to establish cause-and-effectrelationships between exposures of concern and cancer. Thisrequirement is not included in theproposed guidelines.
On May 6, 1996, EPA held a forum on the proposed guidelines. Atthis forum, EPA staff wasasked why statistical significance was not included as arequirement for determining cause-andeffect relationships. EPA staff denied that the statisticalsignificance requirement had beenomitted.
Do the proposed guidelines require that epidemiologic studies bestatistically significant beforethey may be used to support conclusions that cause-and-effectrelationships exist betweenexposures of concern and cancer.
WHAT IS STATISTICAL SIGNIFICANCE AND WHY IS IT SIGNIFICANT?
In its 1991 report entitled Environmental Epidemiology: PublicHealth and HazardousWastes , the National Academy of Sciences stated that(3)Historically, discussions on causality [based on epidemiology]have proceeded once astatistically significant relationship between a potential casualfactor and a disease has beenfound, as is discussed below...The requirement that a finding bestatistically significant has beena convention of epidemiologic research. If results have alikelihood of only 5 percent or less ofoccurring by chance, then they are usually considered statisticallysignificant, as measured by anumber of customary tests, such as p and t values.
Thus, statistical significance is a traditional and standard toolto rule out luck or chance as beingthe factor that has caused the observed results of an epidemiologicstudy.
STATISTICAL SIGNIFICANCE IS REQUIRED BY THE 1986GUIDELINES.
According to the 1986 guidelines:Three criteria must be met before a causal association can beinferred between exposureand cancer in humans:
1. There is no identified bias that could explain theassociation.
2. The possibility of confounding has been considered and ruled outas explaining theassociation.
3. The association is unlikely to be due tochance.
The latter criteria is statistical significance.
Statistical significance is not required by the 1996 proposedguidelines.
In relevant part, the 1996 guidelines propose that(4)18.104.22.168. Criteria for Causality. A causal interpretation isenhanced for studies to the extentthat they meet the criteria described below. None of the criteriais conclusive by itself, and theonly criterion that is essential is the temporalrelationship...
Temporal relationship: The development ofcancers requirecertain latency periods, and while latency periods vary, existenceof such periods is generallyacknowledged. Thus, the disease has to occur within a biologicallyreasonable time after initialexposure. This feature must be present if causality is to beconsidered.
Consistency: Associations occur in severalindependentstudies of a similar exposure in different populations, orassociations occur consistently fordifferent subgroups in the same study. This feature usuallyconstitutes strong evidence for acausal interpretation when the same bias or confounding is not alsoduplicated acrossstudies.
Magnitude of the association: A causalrelationship is morecredible when the risk estimate is large and precise (narrowconfidence intervals).
Biological gradient: The risk ratio (i.e.,the ratio of the riskof disease or death among the exposed to the risk of the unexposed)increases with increasingexposure or dose. A strong dose response relationship acrossseveral categories of exposure,latency, and duration is supportive for causality given thatconfounding is unlikely to becorrelated with exposure. The absence of a dose responserelationship, however, is not by itselfevidence against a causal relationship.
Specificity of the association: Thelikelihood of a causalinterpretation is increased if an exposure produces a specificeffect (one or more tumor types alsofound in other studies) or if a given effect has a uniqueexposure.
Biological plausibility: The associationmakes sense interms of biological knowledge. Information is considered fromanimal toxicology, toxicokinetics,structure-activity relationship analysis, and short-term studies ofthe agent's influence on eventsin the carcinogenic process considered.
Coherence: The cause-and-effectinterpretation is in logicalagreement with what is known about the natural history and biologyof the disease, i.e., theentire body of knowledge about the agent.
Although the number of criteria have increased from three in the1986 guidelines to seven in theproposed guidelines, the requirement of statistical significance isnot included in the proposedguidelines.
THE PROPOSED GUIDELINES MISLEAD THE PUBLIC ABOUT THE STATISTICALSIGNIFICANCE REQUIREMENT.
The proposed guidelines contain language in several places thatgives the misleading impressionthat statistical significance is included in the guidelines.Proposed section 22.214.171.124. Criteriafor Assessing Adequacy of Epidemiologic Studies , states(5)Statistical Considerations. The analysis applies appropriatestatistical methods to ascertainwhether or not there is any significant association betweenexposure and effects. A descriptionof the method or methods should include the reasons for theirselection. Statistical analyses ofthe potential effects of bias or confounding factors are part ofaddressing the significance of anassociation, or lack of one, and whether a study is able to detectany effect.
The analysis augments examination of the results for the wholepopulation with exploration ofthe results for groups with comparatively greater exposure or timesince first exposure. This maysupport identifying an association or establishing a dose responsetrend. When studies show noassociation, such exploration may apply to determining an upperlimit on potential human riskfor consideration alongside results of animal tumor effectsstudies.
This language mentions doing statistical analyses only todetermine whetherstatistical significance exists. This language does not requirethat statistical significance existbefore concluding that a causal relationship exists.
Also, section 2.6.1 Weight of Evidence Evaluation for PotentialHuman Carcinogenicity, states(6)Human Evidence. Analyzing the contribution of evidence from abody of human datarequires examining available studies and weighing them in thecontext of well-accepted criteriafor causation (see section 2.2.1). A judgment is made about howclosely they satisfy thesecriteria, individually and jointly, and how far they deviate fromthem. Existence of temporalrelationships, consistent results in independent studies, strongassociation, reliable exposure data,presence of dose-related responses, freedom from biases andconfounding factors, and high levelof statistical significance are among the factors leading toincreased confidence in a conclusionof causality.
Although statistical significance is mentioned as a factor inincreasing confidence in a conclusionof causality, this language does not make statistical significancea basic requirement fordetermining causality.
Finally, the proposed guidelines include Figure 2-1 entitled Human Evidence Factors is presented to summarize theabove-quoted section 2.6.1.(7).
Human Evidence Factors
Number of independent studies with consistent results
Most causal criteria satisfied:Temporal relationship
Reliable exposure data
Dose response relationship
Freedom from bias and confounding
High statistical significance
Equally well designed and conducted studies with null results
Few causal crieria satisfied
Figure 2-1. Factors for Weighing Human Evidence
Copyright © 1996
StevenJ. Milloy. All rights reserved. Site developed and hosted by WestLakeSolutions, Inc.