EVALUATING THE LITERATURE ON PREOPERATIVE ANDROGEN STIMULATION FOR HYPOSPADIAS REPAIR USING THE FRAGILITY INDEX: WHICH CAN WE TRUST- RANDOMIZED CONTROLLED TRIALS OR OBSERVATIONAL STUDIES?
Bruce Li, MD-C, Isaac Kong, BMSc, MD-C, Melissa McGrath, BASc, Luis H. Braga, MD, PhD.
McMaster University, Hamilton, ON, Canada.
BACKGROUND: Hypospadias publications involving preoperative androgen stimulation (PAS) suffer from small sample sizes and lack of power to claim robust conclusions. Small variations in the study results represent a large proportion of their sample and may completely change the statistical significance, making the conclusions drawn unreliable. Fragility index (FI) is the number of additional events needed to occur in either the control or experimental group in order to lose statistical significance. Typically, appraisal of literature involves assessment of power and p-value within the context of study design and methodology. We aimed to correlate the FI to both power and p-value in order to establish the FI as another statistical tool to evaluate the robustness of study results. We calculated the FI for randomized controlled trials (RCTs) and observational studies investigating the effect of PAS on complications post-hypospadias repair.
METHODS: A comprehensive search of MEDLINE, EMBASE, and grey literature(ESPU and SPU abstracts), to identify RCTs and observational studies investigating the effect of PAS on complications post-hypospadias repair between 1990 and 2019, was conducted. The FI was calculated for each study. Postoperative complications were defined as: fistula, stricture/stenosis, diverticula, and dehiscence. The relative risk (RR), 95% confidence intervals (CI), corresponding p-values and power were calculated. CI upper limits that overlapped 1 up to 1.2 were considered equal to 1 FI. CI lower limits that overlapped 1 down to 0.8 were considered equal to 1.
RESULTS: Fifteen studies qualified for inclusion and their 95% CI's were plotted (Figure 1). When stratified based on p-value groups (<0.0001, 0.0001 - 0.001, 0.0011 - 0.01, 0.011 - 0.05), the mean FI was found to be 22, 12, N/A, and 2, respectively (Figure 2A). The post-hoc power calculation demonstrated a significantly positive correlation with the corresponding FI (R 2 = 0.5543, p < 0.01)(Figure 2B). The mean FI of observational studies was similar to that of RCTs (3.7 vs. 1, p=0.44).
CONCLUSION: The FI was found to be positively correlated with power and negatively with p-value, indicating its ability to assess the robustness of study results.
RCTs showed either weak protective effect or no harmful influence of PAS on complications post-hypospadias repair, but their FIs were very small, indicating that their results were not robust. Even though a few observational studies showed strong statistical significance based on power, p-value, and FI, their results should be interpreted with caution, as these types of study design are likely affected by confounding factors and biases.
In summary, FI should be considered when appraising hypospadias literature as it provides important information regarding the robustness of the results. PAS literature is insufficient from either a statistical or a methodological standpoint to draw strong conclusions, therefore caution is warranted before changing clinical practice. 
Back to 2019 Abstracts
