Andrew D. Asher, Kristin A. Briney, Abigail H. Goben
{"title":"Valid questions: the development and evaluation of a new library learning analytics survey","authors":"Andrew D. Asher, Kristin A. Briney, Abigail H. Goben","doi":"10.1108/pmm-04-2023-0009","DOIUrl":null,"url":null,"abstract":"PurposeThis article describes the development processes, sampling and analysis practices and the assessment of reliability and validity of a new 0survey that sought to evaluate undergraduate students' perceptions and expectations related to privacy and library participation in learning analytics studies. This article provides other researchers with information required to independently evaluate the survey's efficacy, as well as guidance for designing other surveys.Design/methodology/approachFollowing question development, pre-survey validity assessments were made using subject matter expert panel review and cognitive interviews. Post-hoc analysis of survey construct reliability was evaluated using the Omega coefficient, while exploratory factor analysis was utilized to assess construct validity. Survey design limitations and potential bias effects are also examined.FindingsThe survey exhibited a high level of reliability among research constructs, while the exploratory factor analysis results suggested that survey constructs contained multiple conceptual elements that should be measured separately for more nuanced analysis.Practical implicationsThis article provides a model for other researchers wishing to re-use the survey described or develop similar surveys.Social implicationsAs learning analytics interest continues to expand, engaging with the subjects, in this case students, of analysis is critical. Researchers need to ensure that captured measurements are appropriately valid in order to accurately represent the findings.Originality/valueThis survey is one of very few addressing library learning analytics that has undergone extensive validity analysis of the conceptual constructs.","PeriodicalId":44583,"journal":{"name":"Performance Measurement and Metrics","volume":" ","pages":""},"PeriodicalIF":1.8000,"publicationDate":"2023-07-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Performance Measurement and Metrics","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1108/pmm-04-2023-0009","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"INFORMATION SCIENCE & LIBRARY SCIENCE","Score":null,"Total":0}
引用次数: 0
Abstract
PurposeThis article describes the development processes, sampling and analysis practices and the assessment of reliability and validity of a new 0survey that sought to evaluate undergraduate students' perceptions and expectations related to privacy and library participation in learning analytics studies. This article provides other researchers with information required to independently evaluate the survey's efficacy, as well as guidance for designing other surveys.Design/methodology/approachFollowing question development, pre-survey validity assessments were made using subject matter expert panel review and cognitive interviews. Post-hoc analysis of survey construct reliability was evaluated using the Omega coefficient, while exploratory factor analysis was utilized to assess construct validity. Survey design limitations and potential bias effects are also examined.FindingsThe survey exhibited a high level of reliability among research constructs, while the exploratory factor analysis results suggested that survey constructs contained multiple conceptual elements that should be measured separately for more nuanced analysis.Practical implicationsThis article provides a model for other researchers wishing to re-use the survey described or develop similar surveys.Social implicationsAs learning analytics interest continues to expand, engaging with the subjects, in this case students, of analysis is critical. Researchers need to ensure that captured measurements are appropriately valid in order to accurately represent the findings.Originality/valueThis survey is one of very few addressing library learning analytics that has undergone extensive validity analysis of the conceptual constructs.
期刊介绍:
■Quantitative and qualitative analysis ■Benchmarking ■The measurement and role of information in enhancing organizational effectiveness ■Quality techniques and quality improvement ■Training and education ■Methods for performance measurement and metrics ■Standard assessment tools ■Using emerging technologies ■Setting standards or service quality