{"title":"PP 1/333 (1) Adoption of digital technology for data generation for the efficacy evaluation of plant protection products","authors":"","doi":"10.1111/epp.13037","DOIUrl":null,"url":null,"abstract":"<p><b>Specific scope:</b> This Standard describes the validation, verification, and calibration of digital technologies that may be used to assess the efficacy of plant protection products (PPP). Currently, efficacy data are collected through human observation or other documented methods of assessment in the Good Experimental Practice (GEP) system as described in PP 1/181 <i>Conduct and reporting of efficacy evaluation trials, including good experimental practice</i>.1</p><p>Hardware or sensors which directly produce a measurement (e.g. scales for weighing, thermometers) are out of the scope of this Standard, as their verification and calibration are already covered in the GEP system. New technologies with parameters that are currently not covered by EPPO PP1 Standards are also out of this scope but could be covered in future by specific EPPO Standards.</p><p><b>Specific approval and amendment:</b> First approved in 2024–09.</p><p>The development and integration of digital technology is growing across a wide range of industries including agriculture. In crop protection and plant phenotyping, digital technologies are already well established in research and at grower level. Usage of digital technology for the assessment of the efficacy of plant protection products is also rapidly increasing.</p><p>This Standard focuses on how digital technologies used in efficacy trials can be accepted within GEP systems and by regulators. The Standard also includes processes to validate, verify, and calibrate digital technologies, relevant for the GEP system.</p><p>When used in efficacy evaluation trials, digital technologies should produce an outcome which is comparable to the data being currently collected by human observation or by other methods currently accepted in the GEP system. The data obtained by human observation which are used as comparisons to validate or verify data obtained by digital technologies can be referred to as reference values (sometimes referred to as ‘ground truth’). Reference value data enables validation of algorithms in the development stage and verification of digital technology during use.</p><p>As described in PP 1/152 <i>Design and analysis of evaluation of trials</i>, the qualities to be considered in observations for efficacy evaluations are: accuracy, reliability, precision, sensitivity, repeatability and reproducibility. These qualities should also be considered in the evaluation of digital technologies used in the GEP system.</p><p>Validation is a crucial process which serves to develop and assess the accuracy and reliability of the digital technology and to define the specification of any hardware required to generate the data for the algorithm, model or software. Validation is the first step of the digital technology conducted under development conditions to show that it works for the intended use.</p><p>The validation process should use a known or pre-defined dataset or samples and compare the results from the digital technology with the assessment results from a human observation or from other methods currently accepted in the GEP system (the reference value). This should be carried out by experts in that type of assessment and should be conducted in controlled situations. The development of the algorithm, model or software may or may not include an element of machine learning or artificial intelligence.</p><p>The validation procedure, the dataset and samples used for the development of the model/algorithm/software and the validation criteria will be different for each technology and will vary depending on the stage of development and are defined by the developer.</p><p>No further guidance on validation is given in this Standard, since validation is the first step of the digital technology conducted under development conditions, and it is not within scope of GEP.</p><p>Verification is the process to show that the digital technology is robust enough to be used for efficacy assessment in the GEP system. The data obtained by the digital technology will be compared with the data obtained by human observation or by other methods currently accepted in the GEP system (i.e. the reference value). The data forming the basis for the verification should be generated under GEP conditions.</p><p>The digital technology needs to be tested by different users independent of the technology developer and comply with the relevant instructions provided by the digital technology developer.</p><p>Based on the verification, the digital technology provider defines the intended use and the conditions under which accuracy of the digital technology is guaranteed and relevant limitations identified.</p><p>Compiling a verification report, considering the requirements to prove accuracy and identifying the different responsibilities in the context of GEP are important elements of the verification process.</p><p>Within a GEP system, evidence is required that equipment used in a GEP accredited trial has been subject to an appropriate calibration procedure on a regular basis, with relevant details recorded as evidence that the equipment was operating correctly and within defined parameters at the time used in the trial.</p><p>Calibration is also a required procedure with digital technologies to ensure that the hardware used works accurately within the required specification to feed data of the required quality into the model, algorithm or software.</p><p>Calibration is the process of evaluating and adjusting the precision and accuracy of measurement of the hardware, to ensure it is within the acceptable range. The proper calibration of the hardware ensures that valid data is produced for GEP efficacy trials. The relevant manufacturers' instruction on calibration of the hardware should be followed in conjunction with any additional specific requirements from the provider of the algorithm/model/software.</p><p>The calibration procedure should be reflected in a relevant standard operational procedure of the trial facility.</p><p>A record should be made of the conduct of the calibration, noting relevant information including the identification of the hardware equipment being calibrated and the version of the algorithm/model/software being used. Such information should also be recorded as part of the assessment data when digital technology are used for an assessment in a GEP trial. In case of results outside the range prescribed by the provider of the hardware or the algorithm/model/software the provider(s) should be contacted and the tool verified again.</p><p>In addition to equipment calibration, checking the accuracy of the digital technology also needs to be performed regularly, to confirm that it continues to match as minimum the accepted way of assessment in the GEP system e.g., accuracy of human observation, and there is no degradation in performance.</p><p>The technology provider is responsible for ensuring that the software is functioning properly and up to date and to make the most recent software version available to the user. The user is responsible for using the most appropriate and compatible version. Software versioning should be documented in the verification report.</p><p>In case of significant deviation of the digital technology measurement from the expected value, the provider should be contacted and the technology verified again.</p>","PeriodicalId":34952,"journal":{"name":"EPPO Bulletin","volume":"55 1","pages":"14-19"},"PeriodicalIF":0.0000,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://onlinelibrary.wiley.com/doi/epdf/10.1111/epp.13037","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"EPPO Bulletin","FirstCategoryId":"1085","ListUrlMain":"https://onlinelibrary.wiley.com/doi/10.1111/epp.13037","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"Agricultural and Biological Sciences","Score":null,"Total":0}
引用次数: 0
Abstract
Specific scope: This Standard describes the validation, verification, and calibration of digital technologies that may be used to assess the efficacy of plant protection products (PPP). Currently, efficacy data are collected through human observation or other documented methods of assessment in the Good Experimental Practice (GEP) system as described in PP 1/181 Conduct and reporting of efficacy evaluation trials, including good experimental practice.1
Hardware or sensors which directly produce a measurement (e.g. scales for weighing, thermometers) are out of the scope of this Standard, as their verification and calibration are already covered in the GEP system. New technologies with parameters that are currently not covered by EPPO PP1 Standards are also out of this scope but could be covered in future by specific EPPO Standards.
Specific approval and amendment: First approved in 2024–09.
The development and integration of digital technology is growing across a wide range of industries including agriculture. In crop protection and plant phenotyping, digital technologies are already well established in research and at grower level. Usage of digital technology for the assessment of the efficacy of plant protection products is also rapidly increasing.
This Standard focuses on how digital technologies used in efficacy trials can be accepted within GEP systems and by regulators. The Standard also includes processes to validate, verify, and calibrate digital technologies, relevant for the GEP system.
When used in efficacy evaluation trials, digital technologies should produce an outcome which is comparable to the data being currently collected by human observation or by other methods currently accepted in the GEP system. The data obtained by human observation which are used as comparisons to validate or verify data obtained by digital technologies can be referred to as reference values (sometimes referred to as ‘ground truth’). Reference value data enables validation of algorithms in the development stage and verification of digital technology during use.
As described in PP 1/152 Design and analysis of evaluation of trials, the qualities to be considered in observations for efficacy evaluations are: accuracy, reliability, precision, sensitivity, repeatability and reproducibility. These qualities should also be considered in the evaluation of digital technologies used in the GEP system.
Validation is a crucial process which serves to develop and assess the accuracy and reliability of the digital technology and to define the specification of any hardware required to generate the data for the algorithm, model or software. Validation is the first step of the digital technology conducted under development conditions to show that it works for the intended use.
The validation process should use a known or pre-defined dataset or samples and compare the results from the digital technology with the assessment results from a human observation or from other methods currently accepted in the GEP system (the reference value). This should be carried out by experts in that type of assessment and should be conducted in controlled situations. The development of the algorithm, model or software may or may not include an element of machine learning or artificial intelligence.
The validation procedure, the dataset and samples used for the development of the model/algorithm/software and the validation criteria will be different for each technology and will vary depending on the stage of development and are defined by the developer.
No further guidance on validation is given in this Standard, since validation is the first step of the digital technology conducted under development conditions, and it is not within scope of GEP.
Verification is the process to show that the digital technology is robust enough to be used for efficacy assessment in the GEP system. The data obtained by the digital technology will be compared with the data obtained by human observation or by other methods currently accepted in the GEP system (i.e. the reference value). The data forming the basis for the verification should be generated under GEP conditions.
The digital technology needs to be tested by different users independent of the technology developer and comply with the relevant instructions provided by the digital technology developer.
Based on the verification, the digital technology provider defines the intended use and the conditions under which accuracy of the digital technology is guaranteed and relevant limitations identified.
Compiling a verification report, considering the requirements to prove accuracy and identifying the different responsibilities in the context of GEP are important elements of the verification process.
Within a GEP system, evidence is required that equipment used in a GEP accredited trial has been subject to an appropriate calibration procedure on a regular basis, with relevant details recorded as evidence that the equipment was operating correctly and within defined parameters at the time used in the trial.
Calibration is also a required procedure with digital technologies to ensure that the hardware used works accurately within the required specification to feed data of the required quality into the model, algorithm or software.
Calibration is the process of evaluating and adjusting the precision and accuracy of measurement of the hardware, to ensure it is within the acceptable range. The proper calibration of the hardware ensures that valid data is produced for GEP efficacy trials. The relevant manufacturers' instruction on calibration of the hardware should be followed in conjunction with any additional specific requirements from the provider of the algorithm/model/software.
The calibration procedure should be reflected in a relevant standard operational procedure of the trial facility.
A record should be made of the conduct of the calibration, noting relevant information including the identification of the hardware equipment being calibrated and the version of the algorithm/model/software being used. Such information should also be recorded as part of the assessment data when digital technology are used for an assessment in a GEP trial. In case of results outside the range prescribed by the provider of the hardware or the algorithm/model/software the provider(s) should be contacted and the tool verified again.
In addition to equipment calibration, checking the accuracy of the digital technology also needs to be performed regularly, to confirm that it continues to match as minimum the accepted way of assessment in the GEP system e.g., accuracy of human observation, and there is no degradation in performance.
The technology provider is responsible for ensuring that the software is functioning properly and up to date and to make the most recent software version available to the user. The user is responsible for using the most appropriate and compatible version. Software versioning should be documented in the verification report.
In case of significant deviation of the digital technology measurement from the expected value, the provider should be contacted and the technology verified again.
EPPO BulletinAgricultural and Biological Sciences-Horticulture
CiteScore
1.80
自引率
0.00%
发文量
70
期刊介绍:
As the official publication of the European and Mediterranean Plant Protection Organization, the EPPO Bulletin publishes research findings on all aspects of plant protection, but particularly those of immediate concern to government plant protection services. Papers are published in English and French, with summaries also in Russian.