Follow-up by mail in clinical trials: does questionnaire length matter?

Abstract

In large clinical trials where outcome assessment is possible using questionnaires, it may be more cost-effective to mail them to patients than to conduct interviews in-person. However, nonresponse to mailed questionnaires reduces the effective sample size and can introduce bias. We conducted a systematic review and meta-analysis of randomized controlled trials evaluating the effect of questionnaire length on response rates. We searched 14 electronic bibliographic databases, the reference lists of relevant trials, and we contacted the authors of eligible trials to ask about unpublished data. For each trial identified, we used logistic regression to estimate the odds ratio for response per one page increase in the number of pages included in the questionnaire. We pooled the regression coefficients in a random effects meta-analysis. Heterogeneity among the coefficients was assessed using a chi-square test at a 5% significance level. We specified a priori that the reduction in the odds of response per one page increase would be greatest among trials comparing relatively short questionnaires. We used meta regression to examine the relationships between the regression coefficients, the length of the questionnaires used in each trial, and other study characteristics. A total of 38 randomized controlled trials were identified where participants were allocated to questionnaires of differing lengths and where the number of pages used was known. There was significant heterogeneity between the regression coefficients estimated from each trial. In meta regression, most of the heterogeneity was explained by variation in the length of the questionnaires used in each trial. Among trials in which the shortest questionnaire was a postcard, the odds of response were more than halved for each additional page used (0.39; 95% CI 0.34 to 0.45). In the remaining trials, pooled effect sizes were much smaller. In trials of one page compared with either two or three pages, the odds of response per one page increase was 1.01 (95% CI 0.82 to 1.24). For one page compared with four or more pages, and for two or more pages compared with longer alternatives, the odds ratios per one page increase were 0.90 (95% CI 0.83 to 0.98) and 0.98 (95% CI 0.96 to 0.99), respectively. There were no statistically significant associations between trial results and other study characteristics. It appears that response can be increased by using a shorter questionnaire. Moderate changes to the length of shorter questionnaires will be more effective than moderate changes to the length of longer questionnaires. If a choice of follow-up questionnaire exists for a clinical trial, the shorter one should be used. If a new follow-up questionnaire is to be designed, it should be made as short as possible without compromising the data collection requirements of the trial.