Comparison of heterogeneity measures in meta-analysis

Abstract

Background: Heterogeneity assessment is critical in meta-analysis, as it determines the appropriateness of combining studies and affects result reliability. Cochran’s Q is the traditional test, nevertheless, it has low statistical power, so many researchers resort to using heterogeneity measures to quantify the heterogeneity. Aim: This article aims to compare the performance of the most commonly used heterogeneity measures through simulation. Materials and Methods: We compared the performance of four heterogeneity measures (!!, !!, !!, H) across various homogeneous and heterogeneous patient-event probabilities [P P! E! and P P! E! ], various sample sizes (n) and number of studies (k), using RMSE (Root mean squared error) and BIAS values in simulation scenarios. Additionally, Cochran’s Q Type-I error rate and power were evaluated using the same simulation scenarios. Results: (Equation Presented) H outperformed other measures in large samples, while (Equation Presented) were preferable for small studies. Conclusion: Researchers can use the simulation results from this study to select an appropriate heterogeneity measure for their meta-analysis work. This approach is expected to prevent time loss due to unnecessary subgroup analyses in situations where heterogeneity appears to be present but is actually absent.