Comparison of the results of in-person and mobile phone surveys for a health facility assessment in Tajikistan: a validation study protocol

Abstract

Health facility assessments provide important data to measure the quality of health services delivered to populations. These assessments are comprehensive, resource intensive, and periodic to inform medium- to-longer-term policies. However, in absence of other reliable data sources, country decision makers often rely on outdated data to address service delivery challenges that change more frequently. High-frequency phone surveys are a potential option to improve the efficiency and timeliness of collecting time-sensitive service delivery indicators in-between comprehensive in-person assessments. The objectives of this study are to assess the reliability, concurrent criterion validity, and non-response rates in a rapid phone-based health facility assessment developed by the Global Financing Facility’s FASTR initiative compared to a comprehensive in-person health facility assessment developed by the World Bank’s Service Delivery Indicators Health Program. The in-person survey and corresponding in-person item verification will serve as the gold standard. Both surveys will be administered to an identical sample of 500 health facilities in Tajikistan using the same data collection entity. To assess reliability, percent agreement, Cohens Kappa, and prevalence and bias adjusted Kappa will be calculated. To assess concurrent criterion validity, sensitivity and specificity will be calculated, with a cut-off of .7 used for adequate validity. The study will further compare response rates and dropout rates of both surveys using simple t-tests and balance tests to identify if the characteristics of the phone-based and in-person survey samples are similar after accounting for any differences in survey response rates. The results of this study will provide important insights into the reliability and validity of phone-based data collection approaches for health facility assessments. This is critical as Ministries of Health seek to establish and sustain more continuous data collection, analysis, and use of health facility-level data to complement periodic in-person assessments to improve the quality of services provided to their populations.