Hong Sun, Xin Zhou, Peiwen Zhang, Xin Liu, Yunsong Lu, Hong Huang, Wuyang Song
{"title":"Competency-based assessment of pilots’ manual flight performance during instrument flight training","authors":"Hong Sun, Xin Zhou, Peiwen Zhang, Xin Liu, Yunsong Lu, Hong Huang, Wuyang Song","doi":"10.1007/s10111-023-00737-3","DOIUrl":null,"url":null,"abstract":"Abstract A pilot's control behavior has a direct impact on aviation safety. To adapt to new flight training reforms, a competency-based assessment method is proposed in this paper. First, assessment scenarios, key competencies and observable behaviours were determined. Then, observable behavior assessment indexes were developed based on standard operating procedures and practical examination standards. Flight training data were analyzed based on curve similarity theory and a time series fast retrieval algorithm, and the assessment indexes were scored. The competency assessment method was established by quantifying “HOW MANY” and “HOW OFTEN” observable behaviors according to the assessment indexes score results. Finally, the fuzzy C-means clustering algorithm was used to classify the assessment values, and assessment results were obtained. A teardrop pattern procedure was used to test the reliability of the assessment method. The results indicate that the correlation coefficient between the model evaluation and the expert evaluation is 0.947 with a significance level of 0.01, reliably and objectively quantifying manual flight performance and improving the scientific nature of manual flight performance assessment methods. Flight training institutions can use this method to establish diverse assessment scenarios and formulate corresponding assessment indexes based on the characteristics of each pilot to reasonably evaluate the manual flight performance of pilot. Targeted training methods can then be adopted for pilot with different performance levels to improve their competency.","PeriodicalId":49090,"journal":{"name":"Cognition Technology & Work","volume":"242 1","pages":"0"},"PeriodicalIF":2.4000,"publicationDate":"2023-09-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Cognition Technology & Work","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s10111-023-00737-3","RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, INDUSTRIAL","Score":null,"Total":0}
引用次数: 0
Abstract
Abstract A pilot's control behavior has a direct impact on aviation safety. To adapt to new flight training reforms, a competency-based assessment method is proposed in this paper. First, assessment scenarios, key competencies and observable behaviours were determined. Then, observable behavior assessment indexes were developed based on standard operating procedures and practical examination standards. Flight training data were analyzed based on curve similarity theory and a time series fast retrieval algorithm, and the assessment indexes were scored. The competency assessment method was established by quantifying “HOW MANY” and “HOW OFTEN” observable behaviors according to the assessment indexes score results. Finally, the fuzzy C-means clustering algorithm was used to classify the assessment values, and assessment results were obtained. A teardrop pattern procedure was used to test the reliability of the assessment method. The results indicate that the correlation coefficient between the model evaluation and the expert evaluation is 0.947 with a significance level of 0.01, reliably and objectively quantifying manual flight performance and improving the scientific nature of manual flight performance assessment methods. Flight training institutions can use this method to establish diverse assessment scenarios and formulate corresponding assessment indexes based on the characteristics of each pilot to reasonably evaluate the manual flight performance of pilot. Targeted training methods can then be adopted for pilot with different performance levels to improve their competency.
期刊介绍:
Cognition, Technology & Work focuses on the practical issues of human interaction with technology within the context of work and, in particular, how human cognition affects, and is affected by, work and working conditions.
The aim is to publish research that normally resides on the borderline between people, technology, and organisations. Including how people use information technology, how experience and expertise develop through work, and how incidents and accidents are due to the interaction between individual, technical and organisational factors.
The target is thus the study of people at work from a cognitive systems engineering and socio-technical systems perspective.
The most relevant working contexts of interest to CTW are those where the impact of modern technologies on people at work is particularly important for the users involved as well as for the effects on the environment and plants. Modern society has come to depend on the safe and efficient functioning of a multitude of technological systems as diverse as industrial production, transportation, communication, supply of energy, information and materials, health and finance.