Intraclass Correlation Coefficients Typical of Cluster-Randomized Studies
Although this study found that interclass correlation coefficients (ICCs) for patient behaviors and intent-to-change those behaviors is small, they are not trivial. Researchers must take these ICCs into account when planning studies to ensure adequate statistical power.
Researchers in practice-based research networks (PBRNs) often conduct cluster randomized studies that rely on natural clustering at the patient level (i.e., when patients receive care through the same doctor or practice) or at the network level (i.e., when networks draw from the same geography or a similar member selection process). This clustering must be considered during study design, and particularly when setting sample sizes, or it can lead to underestimating variability in outcome measures and overstating differences in treatment effects. ICCs are statistical measures of clustering effects. Here, researchers looked at data obtained from the Prescription for Health program, from 5,042 adult patients and 61 medical practices from eight PBRNs. They estimated ICCs—and confidence intervals—for variables at three different levels of clustering: patients within practices, patients within networks, and practices within networks.
- ICCs greater than 0.1, which indicate substantial clustering, were calculated for some characteristics of patients within practices, including some demographics and some health behaviors (smoking and unhealthy diet).
- At the same patients-within-practices level, ICCs less than 0.1, which suggest diversity, were calculated for intent-to-change behaviors related to diet, exercise, and smoking.
- When corresponding ICCs were calculated at the patients-within-network level, they were generally smaller, suggesting heterogeneity increased at that level.
- At the practices-within-networks level, ICCs varied by characteristic.
The authors conclude even ICCs of less than .01 are not trivial and must be accounted for to avoid reducing the study’s statistical power.