Background: Patient-reported outcome measures (PROMs) have been recognized to reflect patient-centered health domains without clinician bias. However, their reliability and validity relative to clinician-ascertained information have not been thoroughly investigated, particularly on the topic of bladder and bowel symptoms in spina bifida (SB). We assessed the level of agreement between PROMs for neurogenic bladder and bowel symptoms and medical record documentation by clinicians in the ambulatory setting.
Methods: We performed a retrospective cross-sectional study of prospectively collected PROM data from patients ≥12-years-old who completed a modified Neurogenic Bowel Dysfunction Score (NBD) and the Neurogenic Bladder Symptom Score (NBSS) May 2019-March 2024 in a multi-disciplinary SB center. PROMs were completed in English or Spanish independently by the patient or with caregiver assistance. Five items from the PROMs were selected based on relevance and availability of equivalent medical record data: stool frequency, stool incontinence frequency, daytime urine incontinence frequency, daytime urine incontinence amount, and nighttime urine incontinence amount. Patient medical records from the same day as PROM completion were reviewed for clinician documentation of these items. Patients who had no information documented on the same day were excluded. For each item, we calculated the percentage of patients with the same response between PROM and medical record and the corresponding weighted Cohen’s Kappa to assess strength of agreement (0-0.20=slight agreement, 0.21-0.40=fair agreement, 0.41-0.60=moderate agreement, 0.61-0.80= substantial agreement, 0.81-1.0=perfect agreement). We performed subgroup analyses of urinary item agreement in urine-incontinent patients and bowel item agreement in stool-incontinent patients, based on patients with any degree of incontinence reported in either the medical record or PROM.
Results: Of 464 eligible patients approached, 454 returned questionnaires. 331/454 (73%) had clinical data available on the same day as PROM completion and answered at least 1 eligible PROM question for inclusion in this study (55% female, mean age 16.9 years, 50% completed questionnaires independently). Between 219 and 274 patients had data available for each of the five items. Across all observations, agreement between PROM and medical record data ranged from 65%-76% (Kappa 0.39-0.64) (Figure). The item with weakest agreement was the amount of nighttime urine incontinence. The item with strongest agreement was the frequency of daytime urine incontinence. 216 patients with urine incontinence and 118 with stool incontinence were included in the subgroup analyses. Between 116 and 168 urine incontinent patients and 82 and 90 stool incontinent patients had data available for individual items. Compared to the overall cohort, agreement was lower in incontinent patients, (31%-67%, Kappa 0.05-0.41) (Figure), with weakest agreement being frequency of stool incontinence.
Conclusions: Overall, there was moderate agreement for most items between PROM and medical record data. When looking only at patients with urine or stool incontinence, agreement was lower; most items had fair agreement. While agreement between PROM and medical record is simpler for fully continent patients, the subjective nature of incontinence makes it difficult to quantify.
