The intraclass correlation coefficient (ICC) is a fundamental parameter of interest in cluster randomized trials as it can greatly affect statistical power. We compare common methods of estimating the ICC in cluster randomized trials with binary outcomes, with a specific focus on their application to community-based cancer prevention trials with primary outcome of self-reported cancer screening. Using three real data sets from cancer screening intervention trials with different numbers and types of clusters and cluster sizes, we obtained point estimates and 95% confidence intervals for the ICC using five methods: the analysis of variance estimator, the Fleiss-Cuzick estimator, the Pearson estimator, an estimator based on generalized estimating equations and an estimator from a random intercept logistic regression model. We compared estimates of the ICC for the overall sample and by study condition. Our results show that ICC estimates from different methods can be quite different, although confidence intervals generally overlap. The ICC varied substantially by study condition in two studies, suggesting that the common practice of assuming a common ICC across all clusters in the trial is questionable. A simulation study confirmed pitfalls of erroneously assuming a common ICC. Investigators should consider using sample size and analysis methods that allow the ICC to vary by study condition.
Keywords: cancer screening, cluster randomized trials, correlated binary data, intervention trials, intraclass correlation coefficient
In cluster or group randomized trials, clusters of individuals such as primary care practices, geographic regions, families or community organizations are randomized to study conditions. Methodological research on such trials has increased dramatically in recent years as challenging issues are increasingly recognized for such trials [1. 2 ].
A key feature of cluster randomized trials is that outcomes of individuals within a cluster are correlated rather than independent. The intraclass correlation coefficient (ICC), usually denoted ρ. provides a quantitative measure of within-cluster correlation. The ICC is variously defined as the Pearson correlation between two members of the same cluster or the proportion of the total variance in the outcome attributable to the variance between clusters.
The ICC is a fundamental parameter of interest in cluster randomized trials. A cluster randomized trial typically has lower power than an individually randomized trial with the same number of subjects; the decrease in power depends on ρ through the variance inflation factor 1+(m − 1)ρ, where m is average cluster size [2 ]. Estimates of the ICC are needed at the design stage for sample size and power calculations, which are greatly affected by the value of ICC. The method of analysis must also account for correlation of responses. In some situations, the ICC itself may be an object of inference. For these reasons, it is important to have reliable estimation procedures for the ICC.
Studies that randomize geographical communities or primary care practices have become common and have been relatively well studied; study of the ICC and its estimation in trials that randomize other types of clusters have received less attention. Examples include the Korean Healthy Life Study [3 ], in which Korean churches in Los Angeles County, California were randomly assigned to intervention or control conditions, and the outcome, self-reported receipt of hepatitis B testing, was assessed among church members. Another example is the hepatitis B control trial among Cambodian Americans conducted by Taylor el at. [4 ], which randomly sampled households from an electronic database of telephone listings and attempted to recruit one man and one woman from each household, with the primary outcome being self-reported receipt of hepatitis B testing. Further examples of diverse cluster types are in [5
]. The nature of the clusters and outcome measures may affect the ICC. The ICC may be expected to be higher in families or small community-based organizations than in large geographical regions where members of the cluster may have little direct interaction with one another. The ICC may also be related to the outcome variable; e.g. self-reported outcomes and objectively measured outcomes may have different ICCs.
In this paper, we compare methods of estimating the ICC for binary data, with a focus on application of these methods to community-based cluster randomized trials of cancer prevention interventions with self-reported screening outcomes. There is a profusion of point and interval estimators of the ICC for binary data in the literature; examples include Pendergast et al [6 ], Ridout el at.[7 ], Zou and Donner [8 ], Turner et al. [9 ] and Chakraborty et al.[10 ]. A number of authors have compared the performance of various estimators. They include Ridout el at.[7 ], Evans et al. [11 ] and Turner et al. [12 ]. We compare five methods of estimating the ICC for binary data. Three have closed-form asymptotic variance formulae [8 ] and two are based on regression models. Three of these methods have been previously compared [7. 8 ] and our work here to further compare them with estimates from the generalized estimating equation (GEE) model and the random effects logistic model is new. Our work to compare arm-specific ICC estimates to overall ICC estimates by these methods also adds to the literature. We apply the methods to three real data sets from cluster randomized trials to promote cancer screening and compare their point and confidence interval estimates for the ICC. We use simulation studies to compare performance of the methods and discuss the practical implications of our findings for the design and analysis of cluster randomized trials.
Methods of Estimating the ICC
Suppose there are k clusters and the i th cluster has ni individuals. The response of the j th individual in the i th cluster is a binary variable Yij with Yij = 1 for success and Yij = 0 for failure. For example, in the context of the Korean Healthy Life Study, we have Yij =1 if the subject is screened for hepatitis B by six months after baseline interview and Yij = 0 if the subject is not. Let Z i = ∑ j Y i j be the total number of successes from the i th cluster and let N = Σni be the total number of observations in the data set.
The five estimators of the ICC that we consider are: (1) the analysis of variance (ANOVA) estimator, (2) the Fleiss-Cuzick estimator, (3) the Pearson estimator, (4) the GEE estimator, and (5) an estimator from the random intercept logistic model. The first three estimators are based on the common correlation model [7. 8. 13 ], which assumes that the probability of success is the same for all individuals, Pr(Yij = 1) = π for all i and j, and that the responses of subjects from different clusters are independent but responses of any two subjects in the same cluster have a common correlation, Corr (Yij . Yil ) = ρ for j ≠ l. where the value of ρ is the same for all clusters. The formulae for these three estimators are reported in Ridout et al. [7 ].
The ANOVA estimator was originally proposed for continuous data but is also used for binary data. The ANOVA point estimator for the ICC is given by