H TAIT 1,2,3, K BLAZEK 1,2, I JIANG 4, D TUNNICLIFFE 1,2
1Sydney School of Public Health, The University of Sydney, Sydney, Australia, 2Centre For Kidney Research, The Children’s Hospital At Westmead, Sydney, Australia, 3Sydney Medical School, The University of Sydney, Sydney, Australia, 4Faculty of Medicine, The University of New South Wales, Sydney, Australia
Aim: We aimed to describe the risk of bias (RoB) and the extent of small study effects in randomised controlled trials (RCTs) of kidney disease.
Background: RCTs with high RoB or small sample sizes may produce inaccurate effect estimates and lead to inappropriate clinical decisions.
Methods: We searched the Cochrane Database of Systematic Reviews for reviews categorised as Cochrane Topic: Kidney Disease AND Cochrane Group: Kidney and Transplant. Data was extracted for each RCT from included reviews for outcomes including author, year of publication, sample size, RoB (7 domains), and effect sizes (with confidence intervals) for death and other critical outcomes for decision making. Multivariable analysis was conducted using a mixed effects model to compare ‘death’ effects sizes between small and large sample size, compared for effect sizes <1 (beneficial) and >1 (harmful). A conservative estimate of optimal information size of 200 participants per RCT was used. Time series graphs were plotted for the proportion of studies with low RoB in each domain.
Results: From the 180 relevant reviews, 928 RCTs reported ‘death’. Small studies compared to large studies reported a decrease in death estimate of -0.217 (p value <0.00001) and an increase in death estimate of 0.193 (p value <0.00001). Time trends indicate increases in the proportion of low RoB in the domains of sequence generation, allocation concealment, and selective reporting since 2004.
Conclusions: RCTs in nephrology exhibit small study effects in critical outcomes that may impact clinical decision making. However, the quality of RCTs has improved since reporting guidelines and clinical registries have become mandatory. There is still a need for greater attention to trial methods, and further multi-centre collaboration for larger trials.
Bio to come.