评估自动分配的特定工作面试模块。

Annals of Occupational Hygiene Pub Date : 2016-08-01 Epub Date: 2016-06-01 DOI:10.1093/annhyg/mew029
Melissa C Friesen, Qing Lan, Calvin Ge, Sarah J Locke, Dean Hosgood, Lin Fritschi, Troy Sadkowsky, Yu-Cheng Chen, Hu Wei, Jun Xu, Tai Hing Lam, Yok Lam Kwong, Kexin Chen, Caigang Xu, Yu-Chieh Su, Brian C H Chiu, Kai Ming Dennis Ip, Mark P Purdue, Bryan A Bassig, Nat Rothman, Roel Vermeulen
{"title":"评估自动分配的特定工作面试模块。","authors":"Melissa C Friesen,&nbsp;Qing Lan,&nbsp;Calvin Ge,&nbsp;Sarah J Locke,&nbsp;Dean Hosgood,&nbsp;Lin Fritschi,&nbsp;Troy Sadkowsky,&nbsp;Yu-Cheng Chen,&nbsp;Hu Wei,&nbsp;Jun Xu,&nbsp;Tai Hing Lam,&nbsp;Yok Lam Kwong,&nbsp;Kexin Chen,&nbsp;Caigang Xu,&nbsp;Yu-Chieh Su,&nbsp;Brian C H Chiu,&nbsp;Kai Ming Dennis Ip,&nbsp;Mark P Purdue,&nbsp;Bryan A Bassig,&nbsp;Nat Rothman,&nbsp;Roel Vermeulen","doi":"10.1093/annhyg/mew029","DOIUrl":null,"url":null,"abstract":"<p><strong>Objective: </strong>In community-based epidemiological studies, job- and industry-specific 'modules' are often used to systematically obtain details about the subject's work tasks. The module assignment is often made by the interviewer, who may have insufficient occupational hygiene knowledge to assign the correct module. We evaluated, in the context of a case-control study of lymphoid neoplasms in Asia ('AsiaLymph'), the performance of an algorithm that provided automatic, real-time module assignment during a computer-assisted personal interview.</p><p><strong>Methods: </strong>AsiaLymph's occupational component began with a lifetime occupational history questionnaire with free-text responses and three solvent exposure screening questions. To assign each job to one of 23 study-specific modules, an algorithm automatically searched the free-text responses to the questions 'job title' and 'product made or services provided by employer' using a list of module-specific keywords, comprising over 5800 keywords in English, Traditional and Simplified Chinese. Hierarchical decision rules were used when the keyword match triggered multiple modules. If no keyword match was identified, a generic solvent module was assigned if the subject responded 'yes' to any of the three solvent screening questions. If these question responses were all 'no', a work location module was assigned, which redirected the subject to the farming, teaching, health professional, solvent, or industry solvent modules or ended the questions for that job, depending on the location response. We conducted a reliability assessment that compared the algorithm-assigned modules to consensus module assignments made by two industrial hygienists for a subset of 1251 (of 11409) jobs selected using a stratified random selection procedure using module-specific strata. Discordant assignments between the algorithm and consensus assignments (483 jobs) were qualitatively reviewed by the hygienists to evaluate the potential information lost from missed questions with using the algorithm-assigned module (none, low, medium, high).</p><p><strong>Results: </strong>The most frequently assigned modules were the work location (33%), solvent (20%), farming and food industry (19%), and dry cleaning and textile industry (6.4%) modules. In the reliability subset, the algorithm assignment had an exact match to the expert consensus-assigned module for 722 (57.7%) of the 1251 jobs. Overall, adjusted for the proportion of jobs in each stratum, we estimated that 86% of the algorithm-assigned modules would result in no information loss, 2% would have low information loss, and 12% would have medium to high information loss. Medium to high information loss occurred for <10% of the jobs assigned the generic solvent module and for 21, 32, and 31% of the jobs assigned the work location module with location responses of 'someplace else', 'factory', and 'don't know', respectively. Other work location responses had ≤8% with medium to high information loss because of redirections to other modules. Medium to high information loss occurred more frequently when a job description matched with multiple keywords pointing to different modules (29-69%, depending on the triggered assignment rule).</p><p><strong>Conclusions: </strong>These evaluations demonstrated that automatically assigned modules can reliably reproduce an expert's module assignment without the direct involvement of an industrial hygienist or interviewer. The feasibility of adapting this framework to other studies will be language- and exposure-specific.</p>","PeriodicalId":8458,"journal":{"name":"Annals of Occupational Hygiene","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2016-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://sci-hub-pdf.com/10.1093/annhyg/mew029","citationCount":"9","resultStr":"{\"title\":\"Evaluation of Automatically Assigned Job-Specific Interview Modules.\",\"authors\":\"Melissa C Friesen,&nbsp;Qing Lan,&nbsp;Calvin Ge,&nbsp;Sarah J Locke,&nbsp;Dean Hosgood,&nbsp;Lin Fritschi,&nbsp;Troy Sadkowsky,&nbsp;Yu-Cheng Chen,&nbsp;Hu Wei,&nbsp;Jun Xu,&nbsp;Tai Hing Lam,&nbsp;Yok Lam Kwong,&nbsp;Kexin Chen,&nbsp;Caigang Xu,&nbsp;Yu-Chieh Su,&nbsp;Brian C H Chiu,&nbsp;Kai Ming Dennis Ip,&nbsp;Mark P Purdue,&nbsp;Bryan A Bassig,&nbsp;Nat Rothman,&nbsp;Roel Vermeulen\",\"doi\":\"10.1093/annhyg/mew029\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objective: </strong>In community-based epidemiological studies, job- and industry-specific 'modules' are often used to systematically obtain details about the subject's work tasks. The module assignment is often made by the interviewer, who may have insufficient occupational hygiene knowledge to assign the correct module. We evaluated, in the context of a case-control study of lymphoid neoplasms in Asia ('AsiaLymph'), the performance of an algorithm that provided automatic, real-time module assignment during a computer-assisted personal interview.</p><p><strong>Methods: </strong>AsiaLymph's occupational component began with a lifetime occupational history questionnaire with free-text responses and three solvent exposure screening questions. To assign each job to one of 23 study-specific modules, an algorithm automatically searched the free-text responses to the questions 'job title' and 'product made or services provided by employer' using a list of module-specific keywords, comprising over 5800 keywords in English, Traditional and Simplified Chinese. Hierarchical decision rules were used when the keyword match triggered multiple modules. If no keyword match was identified, a generic solvent module was assigned if the subject responded 'yes' to any of the three solvent screening questions. If these question responses were all 'no', a work location module was assigned, which redirected the subject to the farming, teaching, health professional, solvent, or industry solvent modules or ended the questions for that job, depending on the location response. We conducted a reliability assessment that compared the algorithm-assigned modules to consensus module assignments made by two industrial hygienists for a subset of 1251 (of 11409) jobs selected using a stratified random selection procedure using module-specific strata. Discordant assignments between the algorithm and consensus assignments (483 jobs) were qualitatively reviewed by the hygienists to evaluate the potential information lost from missed questions with using the algorithm-assigned module (none, low, medium, high).</p><p><strong>Results: </strong>The most frequently assigned modules were the work location (33%), solvent (20%), farming and food industry (19%), and dry cleaning and textile industry (6.4%) modules. In the reliability subset, the algorithm assignment had an exact match to the expert consensus-assigned module for 722 (57.7%) of the 1251 jobs. Overall, adjusted for the proportion of jobs in each stratum, we estimated that 86% of the algorithm-assigned modules would result in no information loss, 2% would have low information loss, and 12% would have medium to high information loss. Medium to high information loss occurred for <10% of the jobs assigned the generic solvent module and for 21, 32, and 31% of the jobs assigned the work location module with location responses of 'someplace else', 'factory', and 'don't know', respectively. Other work location responses had ≤8% with medium to high information loss because of redirections to other modules. Medium to high information loss occurred more frequently when a job description matched with multiple keywords pointing to different modules (29-69%, depending on the triggered assignment rule).</p><p><strong>Conclusions: </strong>These evaluations demonstrated that automatically assigned modules can reliably reproduce an expert's module assignment without the direct involvement of an industrial hygienist or interviewer. The feasibility of adapting this framework to other studies will be language- and exposure-specific.</p>\",\"PeriodicalId\":8458,\"journal\":{\"name\":\"Annals of Occupational Hygiene\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2016-08-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://sci-hub-pdf.com/10.1093/annhyg/mew029\",\"citationCount\":\"9\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Annals of Occupational Hygiene\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1093/annhyg/mew029\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2016/6/1 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Annals of Occupational Hygiene","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1093/annhyg/mew029","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2016/6/1 0:00:00","PubModel":"Epub","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 9

摘要

目的:在以社区为基础的流行病学研究中,通常使用特定于工作和行业的“模块”来系统地获取有关受试者工作任务的详细信息。模块分配通常由面试官完成,面试官可能没有足够的职业卫生知识来分配正确的模块。在亚洲淋巴肿瘤病例对照研究(“AsiaLymph”)的背景下,我们评估了在计算机辅助个人访谈中提供自动实时模块分配的算法的性能。方法:AsiaLymph的职业成分从终身职业史问卷开始,其中包含自由文本回答和三个溶剂暴露筛选问题。为了将每个工作分配到23个研究特定模块中的一个,算法使用模块特定关键字列表自动搜索“职位”和“雇主制造或提供的产品或服务”问题的自由文本回答,包括5800多个英文,繁体和简体中文关键字。当关键字匹配触发多个模块时,使用分层决策规则。如果没有关键字匹配被确定,如果受试者对三个溶剂筛选问题中的任何一个回答“是”,则分配一个通用溶剂模块。如果这些问题的回答都是“否”,则分配一个工作地点模块,将受试者重新定向到农业、教学、卫生专业、溶剂或工业溶剂模块,或根据地点回答结束该工作的问题。我们进行了可靠性评估,将算法分配的模块与两位工业卫生学家对1251个(11409个)工作的共识模块分配进行了比较,这些工作是使用分层随机选择程序使用模块特定层选择的。通过使用算法分配模块(无,低,中,高),卫生员对算法分配和共识分配(483个作业)之间的不一致分配进行定性审查,以评估因遗漏问题而丢失的潜在信息。结果:最常被分配的模块是工作场所(33%)、溶剂(20%)、农业和食品工业(19%)、干洗和纺织工业(6.4%)模块。在可靠性子集中,1251个作业中有722个(57.7%)的算法分配与专家共识分配模块完全匹配。总体而言,根据每个阶层的工作比例进行调整后,我们估计86%的算法分配模块不会导致信息丢失,2%的模块信息丢失程度较低,12%的模块信息丢失程度中高。结论:这些评估表明,自动分配的模块可以可靠地再现专家的模块分配,而无需工业卫生学家或采访者的直接参与。将这一框架适用于其他研究的可行性将取决于语言和接触的具体情况。
本文章由计算机程序翻译,如有差异,请以英文原文为准。

Evaluation of Automatically Assigned Job-Specific Interview Modules.

Evaluation of Automatically Assigned Job-Specific Interview Modules.

Objective: In community-based epidemiological studies, job- and industry-specific 'modules' are often used to systematically obtain details about the subject's work tasks. The module assignment is often made by the interviewer, who may have insufficient occupational hygiene knowledge to assign the correct module. We evaluated, in the context of a case-control study of lymphoid neoplasms in Asia ('AsiaLymph'), the performance of an algorithm that provided automatic, real-time module assignment during a computer-assisted personal interview.

Methods: AsiaLymph's occupational component began with a lifetime occupational history questionnaire with free-text responses and three solvent exposure screening questions. To assign each job to one of 23 study-specific modules, an algorithm automatically searched the free-text responses to the questions 'job title' and 'product made or services provided by employer' using a list of module-specific keywords, comprising over 5800 keywords in English, Traditional and Simplified Chinese. Hierarchical decision rules were used when the keyword match triggered multiple modules. If no keyword match was identified, a generic solvent module was assigned if the subject responded 'yes' to any of the three solvent screening questions. If these question responses were all 'no', a work location module was assigned, which redirected the subject to the farming, teaching, health professional, solvent, or industry solvent modules or ended the questions for that job, depending on the location response. We conducted a reliability assessment that compared the algorithm-assigned modules to consensus module assignments made by two industrial hygienists for a subset of 1251 (of 11409) jobs selected using a stratified random selection procedure using module-specific strata. Discordant assignments between the algorithm and consensus assignments (483 jobs) were qualitatively reviewed by the hygienists to evaluate the potential information lost from missed questions with using the algorithm-assigned module (none, low, medium, high).

Results: The most frequently assigned modules were the work location (33%), solvent (20%), farming and food industry (19%), and dry cleaning and textile industry (6.4%) modules. In the reliability subset, the algorithm assignment had an exact match to the expert consensus-assigned module for 722 (57.7%) of the 1251 jobs. Overall, adjusted for the proportion of jobs in each stratum, we estimated that 86% of the algorithm-assigned modules would result in no information loss, 2% would have low information loss, and 12% would have medium to high information loss. Medium to high information loss occurred for <10% of the jobs assigned the generic solvent module and for 21, 32, and 31% of the jobs assigned the work location module with location responses of 'someplace else', 'factory', and 'don't know', respectively. Other work location responses had ≤8% with medium to high information loss because of redirections to other modules. Medium to high information loss occurred more frequently when a job description matched with multiple keywords pointing to different modules (29-69%, depending on the triggered assignment rule).

Conclusions: These evaluations demonstrated that automatically assigned modules can reliably reproduce an expert's module assignment without the direct involvement of an industrial hygienist or interviewer. The feasibility of adapting this framework to other studies will be language- and exposure-specific.

求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
审稿时长
2 months
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信