PP 1/333 (1) Adoption of digital technology for data generation for the efficacy evaluation of plant protection products

Q3 Agricultural and Biological Sciences
EPPO Bulletin Pub Date : 2024-11-07 DOI:10.1111/epp.13037
{"title":"PP 1/333 (1) Adoption of digital technology for data generation for the efficacy evaluation of plant protection products","authors":"","doi":"10.1111/epp.13037","DOIUrl":null,"url":null,"abstract":"<p><b>Specific scope:</b> This Standard describes the validation, verification, and calibration of digital technologies that may be used to assess the efficacy of plant protection products (PPP). Currently, efficacy data are collected through human observation or other documented methods of assessment in the Good Experimental Practice (GEP) system as described in PP 1/181 <i>Conduct and reporting of efficacy evaluation trials, including good experimental practice</i>.1</p><p>Hardware or sensors which directly produce a measurement (e.g. scales for weighing, thermometers) are out of the scope of this Standard, as their verification and calibration are already covered in the GEP system. New technologies with parameters that are currently not covered by EPPO PP1 Standards are also out of this scope but could be covered in future by specific EPPO Standards.</p><p><b>Specific approval and amendment:</b> First approved in 2024–09.</p><p>The development and integration of digital technology is growing across a wide range of industries including agriculture. In crop protection and plant phenotyping, digital technologies are already well established in research and at grower level. Usage of digital technology for the assessment of the efficacy of plant protection products is also rapidly increasing.</p><p>This Standard focuses on how digital technologies used in efficacy trials can be accepted within GEP systems and by regulators. The Standard also includes processes to validate, verify, and calibrate digital technologies, relevant for the GEP system.</p><p>When used in efficacy evaluation trials, digital technologies should produce an outcome which is comparable to the data being currently collected by human observation or by other methods currently accepted in the GEP system. The data obtained by human observation which are used as comparisons to validate or verify data obtained by digital technologies can be referred to as reference values (sometimes referred to as ‘ground truth’). Reference value data enables validation of algorithms in the development stage and verification of digital technology during use.</p><p>As described in PP 1/152 <i>Design and analysis of evaluation of trials</i>, the qualities to be considered in observations for efficacy evaluations are: accuracy, reliability, precision, sensitivity, repeatability and reproducibility. These qualities should also be considered in the evaluation of digital technologies used in the GEP system.</p><p>Validation is a crucial process which serves to develop and assess the accuracy and reliability of the digital technology and to define the specification of any hardware required to generate the data for the algorithm, model or software. Validation is the first step of the digital technology conducted under development conditions to show that it works for the intended use.</p><p>The validation process should use a known or pre-defined dataset or samples and compare the results from the digital technology with the assessment results from a human observation or from other methods currently accepted in the GEP system (the reference value). This should be carried out by experts in that type of assessment and should be conducted in controlled situations. The development of the algorithm, model or software may or may not include an element of machine learning or artificial intelligence.</p><p>The validation procedure, the dataset and samples used for the development of the model/algorithm/software and the validation criteria will be different for each technology and will vary depending on the stage of development and are defined by the developer.</p><p>No further guidance on validation is given in this Standard, since validation is the first step of the digital technology conducted under development conditions, and it is not within scope of GEP.</p><p>Verification is the process to show that the digital technology is robust enough to be used for efficacy assessment in the GEP system. The data obtained by the digital technology will be compared with the data obtained by human observation or by other methods currently accepted in the GEP system (i.e. the reference value). The data forming the basis for the verification should be generated under GEP conditions.</p><p>The digital technology needs to be tested by different users independent of the technology developer and comply with the relevant instructions provided by the digital technology developer.</p><p>Based on the verification, the digital technology provider defines the intended use and the conditions under which accuracy of the digital technology is guaranteed and relevant limitations identified.</p><p>Compiling a verification report, considering the requirements to prove accuracy and identifying the different responsibilities in the context of GEP are important elements of the verification process.</p><p>Within a GEP system, evidence is required that equipment used in a GEP accredited trial has been subject to an appropriate calibration procedure on a regular basis, with relevant details recorded as evidence that the equipment was operating correctly and within defined parameters at the time used in the trial.</p><p>Calibration is also a required procedure with digital technologies to ensure that the hardware used works accurately within the required specification to feed data of the required quality into the model, algorithm or software.</p><p>Calibration is the process of evaluating and adjusting the precision and accuracy of measurement of the hardware, to ensure it is within the acceptable range. The proper calibration of the hardware ensures that valid data is produced for GEP efficacy trials. The relevant manufacturers' instruction on calibration of the hardware should be followed in conjunction with any additional specific requirements from the provider of the algorithm/model/software.</p><p>The calibration procedure should be reflected in a relevant standard operational procedure of the trial facility.</p><p>A record should be made of the conduct of the calibration, noting relevant information including the identification of the hardware equipment being calibrated and the version of the algorithm/model/software being used. Such information should also be recorded as part of the assessment data when digital technology are used for an assessment in a GEP trial. In case of results outside the range prescribed by the provider of the hardware or the algorithm/model/software the provider(s) should be contacted and the tool verified again.</p><p>In addition to equipment calibration, checking the accuracy of the digital technology also needs to be performed regularly, to confirm that it continues to match as minimum the accepted way of assessment in the GEP system e.g., accuracy of human observation, and there is no degradation in performance.</p><p>The technology provider is responsible for ensuring that the software is functioning properly and up to date and to make the most recent software version available to the user. The user is responsible for using the most appropriate and compatible version. Software versioning should be documented in the verification report.</p><p>In case of significant deviation of the digital technology measurement from the expected value, the provider should be contacted and the technology verified again.</p>","PeriodicalId":34952,"journal":{"name":"EPPO Bulletin","volume":"55 1","pages":"14-19"},"PeriodicalIF":0.0000,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/epp.13037","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"EPPO Bulletin","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/epp.13037","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Agricultural and Biological Sciences","Score":null,"Total":0}
引用次数: 0

Abstract

Specific scope: This Standard describes the validation, verification, and calibration of digital technologies that may be used to assess the efficacy of plant protection products (PPP). Currently, efficacy data are collected through human observation or other documented methods of assessment in the Good Experimental Practice (GEP) system as described in PP 1/181 Conduct and reporting of efficacy evaluation trials, including good experimental practice.1

Hardware or sensors which directly produce a measurement (e.g. scales for weighing, thermometers) are out of the scope of this Standard, as their verification and calibration are already covered in the GEP system. New technologies with parameters that are currently not covered by EPPO PP1 Standards are also out of this scope but could be covered in future by specific EPPO Standards.

Specific approval and amendment: First approved in 2024–09.

The development and integration of digital technology is growing across a wide range of industries including agriculture. In crop protection and plant phenotyping, digital technologies are already well established in research and at grower level. Usage of digital technology for the assessment of the efficacy of plant protection products is also rapidly increasing.

This Standard focuses on how digital technologies used in efficacy trials can be accepted within GEP systems and by regulators. The Standard also includes processes to validate, verify, and calibrate digital technologies, relevant for the GEP system.

When used in efficacy evaluation trials, digital technologies should produce an outcome which is comparable to the data being currently collected by human observation or by other methods currently accepted in the GEP system. The data obtained by human observation which are used as comparisons to validate or verify data obtained by digital technologies can be referred to as reference values (sometimes referred to as ‘ground truth’). Reference value data enables validation of algorithms in the development stage and verification of digital technology during use.

As described in PP 1/152 Design and analysis of evaluation of trials, the qualities to be considered in observations for efficacy evaluations are: accuracy, reliability, precision, sensitivity, repeatability and reproducibility. These qualities should also be considered in the evaluation of digital technologies used in the GEP system.

Validation is a crucial process which serves to develop and assess the accuracy and reliability of the digital technology and to define the specification of any hardware required to generate the data for the algorithm, model or software. Validation is the first step of the digital technology conducted under development conditions to show that it works for the intended use.

The validation process should use a known or pre-defined dataset or samples and compare the results from the digital technology with the assessment results from a human observation or from other methods currently accepted in the GEP system (the reference value). This should be carried out by experts in that type of assessment and should be conducted in controlled situations. The development of the algorithm, model or software may or may not include an element of machine learning or artificial intelligence.

The validation procedure, the dataset and samples used for the development of the model/algorithm/software and the validation criteria will be different for each technology and will vary depending on the stage of development and are defined by the developer.

No further guidance on validation is given in this Standard, since validation is the first step of the digital technology conducted under development conditions, and it is not within scope of GEP.

Verification is the process to show that the digital technology is robust enough to be used for efficacy assessment in the GEP system. The data obtained by the digital technology will be compared with the data obtained by human observation or by other methods currently accepted in the GEP system (i.e. the reference value). The data forming the basis for the verification should be generated under GEP conditions.

The digital technology needs to be tested by different users independent of the technology developer and comply with the relevant instructions provided by the digital technology developer.

Based on the verification, the digital technology provider defines the intended use and the conditions under which accuracy of the digital technology is guaranteed and relevant limitations identified.

Compiling a verification report, considering the requirements to prove accuracy and identifying the different responsibilities in the context of GEP are important elements of the verification process.

Within a GEP system, evidence is required that equipment used in a GEP accredited trial has been subject to an appropriate calibration procedure on a regular basis, with relevant details recorded as evidence that the equipment was operating correctly and within defined parameters at the time used in the trial.

Calibration is also a required procedure with digital technologies to ensure that the hardware used works accurately within the required specification to feed data of the required quality into the model, algorithm or software.

Calibration is the process of evaluating and adjusting the precision and accuracy of measurement of the hardware, to ensure it is within the acceptable range. The proper calibration of the hardware ensures that valid data is produced for GEP efficacy trials. The relevant manufacturers' instruction on calibration of the hardware should be followed in conjunction with any additional specific requirements from the provider of the algorithm/model/software.

The calibration procedure should be reflected in a relevant standard operational procedure of the trial facility.

A record should be made of the conduct of the calibration, noting relevant information including the identification of the hardware equipment being calibrated and the version of the algorithm/model/software being used. Such information should also be recorded as part of the assessment data when digital technology are used for an assessment in a GEP trial. In case of results outside the range prescribed by the provider of the hardware or the algorithm/model/software the provider(s) should be contacted and the tool verified again.

In addition to equipment calibration, checking the accuracy of the digital technology also needs to be performed regularly, to confirm that it continues to match as minimum the accepted way of assessment in the GEP system e.g., accuracy of human observation, and there is no degradation in performance.

The technology provider is responsible for ensuring that the software is functioning properly and up to date and to make the most recent software version available to the user. The user is responsible for using the most appropriate and compatible version. Software versioning should be documented in the verification report.

In case of significant deviation of the digital technology measurement from the expected value, the provider should be contacted and the technology verified again.

PP 1/333 (1) 采用数字技术生成植物保护产品功效评估数据
具体范围:本标准描述了可用于评估植保产品(PPP)功效的数字技术的确认、验证和校准。目前,疗效数据是通过人体观察或在PP 1/181中描述的良好实验规范(GEP)系统中其他记录的评估方法收集的,进行和报告疗效评估试验,包括良好实验规范。直接产生测量的硬件或传感器(如称重秤、温度计)不在本标准的范围内,因为它们的验证和校准已经包含在GEP体系中。具有目前未被EPPO PP1标准涵盖的参数的新技术也不在此范围内,但将来可能会被特定的EPPO标准涵盖。具体批准修改:2024-09年度首次批准。数字技术的发展和整合正在包括农业在内的广泛行业中不断增长。在作物保护和植物表型方面,数字技术已经在研究和种植者层面得到了很好的应用。利用数字技术评估植物保护产品的功效也在迅速增加。本标准侧重于功效试验中使用的数字技术如何在GEP系统内和监管机构中被接受。该标准还包括与GEP系统相关的数字技术的验证、验证和校准过程。当用于疗效评估试验时,数字技术产生的结果应与目前通过人类观察或通过GEP系统中目前接受的其他方法收集的数据相当。通过人类观察获得的数据被用作验证或验证数字技术获得的数据的比较,可以称为参考值(有时称为“基础真值”)。参考值数据可以在开发阶段对算法进行验证,在使用过程中对数字技术进行验证。如PP 1/152所述,试验评估的设计和分析,在疗效评估的观察中要考虑的质量是:准确性、可靠性、精密度、灵敏度、可重复性和可再现性。在评价全球环境方案系统中使用的数字技术时也应考虑到这些品质。验证是一个至关重要的过程,用于开发和评估数字技术的准确性和可靠性,并定义生成算法、模型或软件数据所需的任何硬件规格。验证是在开发条件下进行的数字技术的第一步,以表明它适用于预期用途。验证过程应使用已知或预定义的数据集或样本,并将来自数字技术的结果与来自人类观察的评估结果或来自GEP系统中目前接受的其他方法的评估结果(参考值)进行比较。这应由这类评估的专家进行,并应在受控的情况下进行。算法、模型或软件的开发可能包含也可能不包含机器学习或人工智能的元素。验证程序、用于开发模型/算法/软件的数据集和样本以及验证标准将因每种技术而异,并将根据开发阶段而变化,并由开发人员定义。由于验证是在开发条件下进行的数字技术的第一步,不在GEP的范围内,因此本标准没有给出进一步的验证指导。验证是表明数字技术具有足够的鲁棒性,可以用于GEP系统的有效性评估的过程。通过数字技术获得的数据将与人类观测或GEP系统目前接受的其他方法获得的数据(即参考值)进行比较。构成验证基础的数据应在GEP条件下生成。数字技术需要独立于技术开发人员,由不同的用户进行测试,并遵守数字技术开发人员提供的相关说明。基于验证,数字技术提供商定义了预期用途和保证数字技术准确性的条件,并确定了相关限制。编制验证报告,考虑证明准确性的需求,以及识别GEP背景下的不同职责,是验证过程的重要组成部分。 在GEP系统中,需要有证据表明,在GEP认可的试验中使用的设备已定期接受适当的校准程序,并记录相关细节,以证明设备在试验时正确运行并在规定的参数范围内。校准也是使用数字技术的必要程序,以确保所使用的硬件在所需规格内准确工作,以将所需质量的数据输入模型,算法或软件。校准是评价和调整硬件测量的精度和准确度,以确保其在可接受范围内的过程。正确的硬件校准确保为GEP功效试验提供有效的数据。应遵循相关制造商关于硬件校准的说明,以及算法/模型/软件提供商的任何其他特定要求。校准程序应反映在试验设施的相关标准操作程序中。校准过程应记录下来,注明相关信息,包括被校准硬件设备的标识和所使用的算法/模型/软件的版本。当在全球环境评估试验中使用数字技术进行评估时,这些信息也应作为评估数据的一部分进行记录。如果结果超出硬件或算法/模型/软件提供商规定的范围,则应联系提供商并重新验证工具。除设备校准外,还需要定期检查数字技术的准确性,以确认其至少符合GEP系统中可接受的评估方式,例如人类观察的准确性,并且性能没有下降。技术提供商有责任确保软件正常运行和最新,并向用户提供最新的软件版本。用户有责任使用最合适和兼容的版本。软件版本控制应记录在验证报告中。当数字技术测量值与期望值有明显偏差时,应联系供应商,重新对技术进行验证。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
EPPO Bulletin
EPPO Bulletin Agricultural and Biological Sciences-Horticulture
CiteScore
1.80
自引率
0.00%
发文量
70
期刊介绍: As the official publication of the European and Mediterranean Plant Protection Organization, the EPPO Bulletin publishes research findings on all aspects of plant protection, but particularly those of immediate concern to government plant protection services. Papers are published in English and French, with summaries also in Russian.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信