A V Malervein, A V Kochubey, O Y Bogaevskaya, V V Kochubey, M I Lazechko
{"title":"[评估正畸医生能力的情景任务验证(以合并颌畸形病例为例)]。","authors":"A V Malervein, A V Kochubey, O Y Bogaevskaya, V V Kochubey, M I Lazechko","doi":"10.17116/stomat202510401137","DOIUrl":null,"url":null,"abstract":"<p><strong>The aim the study: </strong>The choice of situational tasks to assess the knowledge of orthodontists about the management of patients with combined jaw deformities.</p><p><strong>Materials and methods: </strong>An expert assessment of 13 situational tasks was carried out in relation to three categories: validity - compliance with real cases in clinical practice, clarity - a complete and adequate description of the case; complexity - the difficulty of solving the problem. The assessment by category was performed on a rating six-point scale based on a calculated integral coefficient (IC) equal to the sum of the ratings by category.</p><p><strong>Results: </strong>The values of the Integral coefficient of the first round of estimates differ significantly, <i>p</i>≤0.001. The Duncan test identifies 9 homogeneous subsets. The estimates of tasks No. 10, 1, 9, 4, 3, 11 simultaneously form 4 groups of subsets for which a second round of evaluation is made. The integral coefficient of the points of the second round of the expert assessment is significantly different, <i>p</i>≤0.001. The Duncan test identifies 2 homogeneous subsets. The experts rated task No. 3 significantly higher (M=10.9). The integral coefficient of the third round of assessments of tasks 3.12.13 is significantly different, <i>p</i>=0.023, «reasonableness» (<i>p</i>=0.815) and «clarity» (<i>p</i>= 0.082) do not differ, the \"complexity\" of task 13 is higher than that of task 3 (<i>p</i>=0.026), differences in the \"complexity\" of tasks 3 and 12 (<i>p</i>=0.209), 12 and 13 (<i>p</i>=0.383) no. In the first round, the reasonableness (3.23±1.34 points), complexity (3.15±1.26), clarity (3.04±1.15) of the tasks did not significantly differ (0.055≤<i>p</i>≤0.422). In the second round, the reasonableness (2.98±0.89) of the tasks is higher than their complexity (2.52±0.86) and clarity (2.54±0.74), (<i>p</i>=0.005 and <i>p</i>=0.003), but the complexity and clarity of the tasks are the same (<i>t</i>=0.18, <i>p</i>=0.860). Reasonableness (5.67±0.48), complexity (5.38±0.49) and clarity (5.62±0.49) do not differ in the third round (0.058≤<i>p</i>≤0.763).</p><p><strong>Conclusion: </strong>The situational tasks selected in the study are distinguished by an integral coefficient of validity, complexity and clarity higher than other evaluated tasks, which allows for a multi-level assessment of the knowledge of orthodontists in the management of patients with combined dental anomalies. Expert assessment is acceptable for validating situational tasks when identifying assessment categories.</p>","PeriodicalId":35887,"journal":{"name":"Stomatologiya","volume":"104 1","pages":"37-41"},"PeriodicalIF":0.0000,"publicationDate":"2025-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"[Validation of situational tasks to assess the competence of orthodontists (using the example of cases of combined jaw deformity)].\",\"authors\":\"A V Malervein, A V Kochubey, O Y Bogaevskaya, V V Kochubey, M I Lazechko\",\"doi\":\"10.17116/stomat202510401137\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>The aim the study: </strong>The choice of situational tasks to assess the knowledge of orthodontists about the management of patients with combined jaw deformities.</p><p><strong>Materials and methods: </strong>An expert assessment of 13 situational tasks was carried out in relation to three categories: validity - compliance with real cases in clinical practice, clarity - a complete and adequate description of the case; complexity - the difficulty of solving the problem. The assessment by category was performed on a rating six-point scale based on a calculated integral coefficient (IC) equal to the sum of the ratings by category.</p><p><strong>Results: </strong>The values of the Integral coefficient of the first round of estimates differ significantly, <i>p</i>≤0.001. The Duncan test identifies 9 homogeneous subsets. The estimates of tasks No. 10, 1, 9, 4, 3, 11 simultaneously form 4 groups of subsets for which a second round of evaluation is made. The integral coefficient of the points of the second round of the expert assessment is significantly different, <i>p</i>≤0.001. The Duncan test identifies 2 homogeneous subsets. The experts rated task No. 3 significantly higher (M=10.9). The integral coefficient of the third round of assessments of tasks 3.12.13 is significantly different, <i>p</i>=0.023, «reasonableness» (<i>p</i>=0.815) and «clarity» (<i>p</i>= 0.082) do not differ, the \\\"complexity\\\" of task 13 is higher than that of task 3 (<i>p</i>=0.026), differences in the \\\"complexity\\\" of tasks 3 and 12 (<i>p</i>=0.209), 12 and 13 (<i>p</i>=0.383) no. In the first round, the reasonableness (3.23±1.34 points), complexity (3.15±1.26), clarity (3.04±1.15) of the tasks did not significantly differ (0.055≤<i>p</i>≤0.422). In the second round, the reasonableness (2.98±0.89) of the tasks is higher than their complexity (2.52±0.86) and clarity (2.54±0.74), (<i>p</i>=0.005 and <i>p</i>=0.003), but the complexity and clarity of the tasks are the same (<i>t</i>=0.18, <i>p</i>=0.860). Reasonableness (5.67±0.48), complexity (5.38±0.49) and clarity (5.62±0.49) do not differ in the third round (0.058≤<i>p</i>≤0.763).</p><p><strong>Conclusion: </strong>The situational tasks selected in the study are distinguished by an integral coefficient of validity, complexity and clarity higher than other evaluated tasks, which allows for a multi-level assessment of the knowledge of orthodontists in the management of patients with combined dental anomalies. Expert assessment is acceptable for validating situational tasks when identifying assessment categories.</p>\",\"PeriodicalId\":35887,\"journal\":{\"name\":\"Stomatologiya\",\"volume\":\"104 1\",\"pages\":\"37-41\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2025-01-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Stomatologiya\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.17116/stomat202510401137\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q4\",\"JCRName\":\"Medicine\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Stomatologiya","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.17116/stomat202510401137","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"Medicine","Score":null,"Total":0}
[Validation of situational tasks to assess the competence of orthodontists (using the example of cases of combined jaw deformity)].
The aim the study: The choice of situational tasks to assess the knowledge of orthodontists about the management of patients with combined jaw deformities.
Materials and methods: An expert assessment of 13 situational tasks was carried out in relation to three categories: validity - compliance with real cases in clinical practice, clarity - a complete and adequate description of the case; complexity - the difficulty of solving the problem. The assessment by category was performed on a rating six-point scale based on a calculated integral coefficient (IC) equal to the sum of the ratings by category.
Results: The values of the Integral coefficient of the first round of estimates differ significantly, p≤0.001. The Duncan test identifies 9 homogeneous subsets. The estimates of tasks No. 10, 1, 9, 4, 3, 11 simultaneously form 4 groups of subsets for which a second round of evaluation is made. The integral coefficient of the points of the second round of the expert assessment is significantly different, p≤0.001. The Duncan test identifies 2 homogeneous subsets. The experts rated task No. 3 significantly higher (M=10.9). The integral coefficient of the third round of assessments of tasks 3.12.13 is significantly different, p=0.023, «reasonableness» (p=0.815) and «clarity» (p= 0.082) do not differ, the "complexity" of task 13 is higher than that of task 3 (p=0.026), differences in the "complexity" of tasks 3 and 12 (p=0.209), 12 and 13 (p=0.383) no. In the first round, the reasonableness (3.23±1.34 points), complexity (3.15±1.26), clarity (3.04±1.15) of the tasks did not significantly differ (0.055≤p≤0.422). In the second round, the reasonableness (2.98±0.89) of the tasks is higher than their complexity (2.52±0.86) and clarity (2.54±0.74), (p=0.005 and p=0.003), but the complexity and clarity of the tasks are the same (t=0.18, p=0.860). Reasonableness (5.67±0.48), complexity (5.38±0.49) and clarity (5.62±0.49) do not differ in the third round (0.058≤p≤0.763).
Conclusion: The situational tasks selected in the study are distinguished by an integral coefficient of validity, complexity and clarity higher than other evaluated tasks, which allows for a multi-level assessment of the knowledge of orthodontists in the management of patients with combined dental anomalies. Expert assessment is acceptable for validating situational tasks when identifying assessment categories.