Hi David! Thanks for your comment and very important points for external validity.

Respondent pool inclusion: To be in the respondent pool for this study, participants: i) had at some point met with a Busara employee and consented to be invited for studies and provided contact numbers (enrollment dates are somewhat evenly spread across the 6 years prior to the study), ii) might have participated in previous studies at Busara (a significant subset of this sample). In these cases, contact details get checked and updated. However, it’s likely the majority did not attend a Busara study within 6 months of this study, so most numbers would be ‘more than 6 months old’.

Thanks for this thought! I think looking at heterogeneity by previous participation is a good idea. We briefly looked at heterogeneity across enrollment date and found significantly less (but still 15%) completion for those enrolled more than 3 years prior to the study. Previous attendance might be even more predictive, and we’ll be sure to look at it as we build a full paper.

As for trust in the organization, the introduction of the message did not explicitly refer to Busara per se. It read “Hello, MSurvey invites you to take this survey. It is free to participate…..“. None of this sample had participated in MSurvey studies prior.

Invalid numbers: ~110 participants out of ~6300 had invalid numbers and were excluded from analysis — this means that completion rates in this analysis could be overstated by ~2pp if you think the appropriate analysis includes invalid numbers — this should be evenly distributed across treatment groups given randomization, so shouldn’t affect treatment effects meaningfully.