Leonor Guiñón, Anna Soler, Rosa María López, Sonia Pajares, José Manuel González de Aledo, Ana Argudo-Ramírez, José Luis Marín, Judit García-Villoria, Ángeles Sahuquillo, Luisa Alvarez
{"title":"新生儿筛查质量保证的评分者间信度评估。","authors":"Leonor Guiñón, Anna Soler, Rosa María López, Sonia Pajares, José Manuel González de Aledo, Ana Argudo-Ramírez, José Luis Marín, Judit García-Villoria, Ángeles Sahuquillo, Luisa Alvarez","doi":"10.11613/BM.2022.030901","DOIUrl":null,"url":null,"abstract":"<p><strong>Introduction: </strong>To ensure the quality of the new-born screening (NBS), our laboratory reviewed the analytical procedure to detect subjective steps that may represent a risk to the patient. Two subjective activities were identified in the extra-analytical phases: the classification of dried blood spots (DBS) according to their quality and the assignment of haemoglobin patterns. To keep these activities under control, inter-rater studies were implemented. This study aimed to evaluate the inter-rater reliability and the effectiveness of the measures taken to improve the agreement between observers, to assure NBS results' quality.</p><p><strong>Materials and methods: </strong>Dried blood spots specimens were used for the inter-rater studies. Ten studies were performed to assess DBS quality classification, and four to assess the assignment of haemoglobin patterns. Krippendorff's alpha test was used to estimate inter-rater reliability. Causes were investigated when alpha values were below 0.80.</p><p><strong>Results: </strong>For both activities, the reliability obtained in the first studies was inadequate. After investigation, we detected that the criterion to classify a DBS as scant was not consolidated, and also a lack of consensus on whether or not to report Bart's haemoglobin depending on its percentage. Alpha estimates became higher once the training was reinforced and a consensus about the appropriate criteria to be applied was reached.</p><p><strong>Conclusion: </strong>Inter-rater reliability assessment helped us to ensure the quality of subjective activities that could add variability to NBS results. Furthermore, the evolution of the alpha value over time allowed us to verify the effectiveness of the measures adopted.</p>","PeriodicalId":9021,"journal":{"name":"Biochemia Medica","volume":null,"pages":null},"PeriodicalIF":3.8000,"publicationDate":"2022-10-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9344869/pdf/","citationCount":"2","resultStr":"{\"title\":\"Inter-rater reliability assessment for the new-born screening quality assurance.\",\"authors\":\"Leonor Guiñón, Anna Soler, Rosa María López, Sonia Pajares, José Manuel González de Aledo, Ana Argudo-Ramírez, José Luis Marín, Judit García-Villoria, Ángeles Sahuquillo, Luisa Alvarez\",\"doi\":\"10.11613/BM.2022.030901\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Introduction: </strong>To ensure the quality of the new-born screening (NBS), our laboratory reviewed the analytical procedure to detect subjective steps that may represent a risk to the patient. Two subjective activities were identified in the extra-analytical phases: the classification of dried blood spots (DBS) according to their quality and the assignment of haemoglobin patterns. To keep these activities under control, inter-rater studies were implemented. This study aimed to evaluate the inter-rater reliability and the effectiveness of the measures taken to improve the agreement between observers, to assure NBS results' quality.</p><p><strong>Materials and methods: </strong>Dried blood spots specimens were used for the inter-rater studies. Ten studies were performed to assess DBS quality classification, and four to assess the assignment of haemoglobin patterns. Krippendorff's alpha test was used to estimate inter-rater reliability. Causes were investigated when alpha values were below 0.80.</p><p><strong>Results: </strong>For both activities, the reliability obtained in the first studies was inadequate. After investigation, we detected that the criterion to classify a DBS as scant was not consolidated, and also a lack of consensus on whether or not to report Bart's haemoglobin depending on its percentage. Alpha estimates became higher once the training was reinforced and a consensus about the appropriate criteria to be applied was reached.</p><p><strong>Conclusion: </strong>Inter-rater reliability assessment helped us to ensure the quality of subjective activities that could add variability to NBS results. Furthermore, the evolution of the alpha value over time allowed us to verify the effectiveness of the measures adopted.</p>\",\"PeriodicalId\":9021,\"journal\":{\"name\":\"Biochemia Medica\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":3.8000,\"publicationDate\":\"2022-10-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9344869/pdf/\",\"citationCount\":\"2\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Biochemia Medica\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.11613/BM.2022.030901\",\"RegionNum\":3,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICAL LABORATORY TECHNOLOGY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Biochemia Medica","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.11613/BM.2022.030901","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICAL LABORATORY TECHNOLOGY","Score":null,"Total":0}
Inter-rater reliability assessment for the new-born screening quality assurance.
Introduction: To ensure the quality of the new-born screening (NBS), our laboratory reviewed the analytical procedure to detect subjective steps that may represent a risk to the patient. Two subjective activities were identified in the extra-analytical phases: the classification of dried blood spots (DBS) according to their quality and the assignment of haemoglobin patterns. To keep these activities under control, inter-rater studies were implemented. This study aimed to evaluate the inter-rater reliability and the effectiveness of the measures taken to improve the agreement between observers, to assure NBS results' quality.
Materials and methods: Dried blood spots specimens were used for the inter-rater studies. Ten studies were performed to assess DBS quality classification, and four to assess the assignment of haemoglobin patterns. Krippendorff's alpha test was used to estimate inter-rater reliability. Causes were investigated when alpha values were below 0.80.
Results: For both activities, the reliability obtained in the first studies was inadequate. After investigation, we detected that the criterion to classify a DBS as scant was not consolidated, and also a lack of consensus on whether or not to report Bart's haemoglobin depending on its percentage. Alpha estimates became higher once the training was reinforced and a consensus about the appropriate criteria to be applied was reached.
Conclusion: Inter-rater reliability assessment helped us to ensure the quality of subjective activities that could add variability to NBS results. Furthermore, the evolution of the alpha value over time allowed us to verify the effectiveness of the measures adopted.
期刊介绍:
Biochemia Medica is the official peer-reviewed journal of the Croatian Society of Medical Biochemistry and Laboratory Medicine. Journal provides a wide coverage of research in all aspects of clinical chemistry and laboratory medicine. Following categories fit into the scope of the Journal: general clinical chemistry, haematology and haemostasis, molecular diagnostics and endocrinology. Development, validation and verification of analytical techniques and methods applicable to clinical chemistry and laboratory medicine are welcome as well as studies dealing with laboratory organization, automation and quality control. Journal publishes on a regular basis educative preanalytical case reports (Preanalytical mysteries), articles dealing with applied biostatistics (Lessons in biostatistics) and research integrity (Research integrity corner).