Which Endpoints Can Be Reliably Assessed in Semi-field Pollinator Species Testing without Estimating False Positive or False Negative? MDD’s and Replicates Issue

Statistical power, number of replicates and experiment complexity of semi-field and field studies on Apis and non-Apis bee species has become a major issue after publication of the draft European Food Safety Authority (EFSA) Guidance on risk assessment of plant protection products (PPP) on bees (Apis mellifera, Bombus spp. and solitary bees). According to this guidance document, field studies have to be designed to be able to detect significance differences as low as 7% for certain endpoints such as reduction in colony size. This will require an immense number of replicates which is obviously not feasible. In the present study, key endpoints such as mortality, termination rate and number of brood cells in honeybee studies, cocoon production and flight activity in solitary bee studies and number of gynes in bumble bee studies (just to mention some of the endpoints considered) in semi-field studies were analyzed, with Apis mellifera, Bombus terrestris and Osmia bicornis during the past five years (2013-2017). The results indicate huge differences in the percentage minimal detectable differences (%MDDs) (MDD expressed as median of control value of the endpoint in percent) depending on endpoint and species tested. For honeybee semi-field studies, the lowest %MDDs recorded were between 10% and 15% for the endpoints foraging, number of brood cells and colony strength. The highest %MDDs were observed for the endpoint termination rate, with a %MDD of almost 50%. For the endpoints in bumble bee semi-field studies the %MDDs varied between 17% for bumble bee colony weight and 53% for average mortality during the exposure period in the tunnel. The %MDD for the number of gynes (young queens) was slightly below 25%. For the semi-field solitary bee test system, the %MDDs for the measured endpoints seem to be lower than those for the other two species tested. The %MDDs for the endpoints hatching of offspring, nest occupation and number of cocoons were 8%, 13% and 14%, respectively. Most of the %MDDs were between 10% and 30% indicating clearly that the currently performed experimental design for the semi-field pollinator studies allowed to determine relatively small effects on key study endpoints. The analysis indicated that for all the three bee species tested, the semi-field test design detected low %MDDs for most of the endpoints. It was also observed that detectable differences between the control and PPP treatments were much lower in semi-field test designs than in field studies with these bee species. The “perfect sample size” really does not exist but test design and statistical analysis can be adapted to lower the %MDDs. Measured and simulated (according to Student’s t-test-distribution) data and results showed that statistical evaluations parameter selection (e.g., alpha value), data transformation (log10) and the number of replicates had a direct effect on the ability of the test design to detect lower or higher %MDD values. It could show that a change of alpha value from 0.05 to 0.1, increases the ability of the studies to detect lower %MDDs. For most of the measured endpoints, increasing the number of replicates e.g., from four to eight, improved the power of the test design by decreasing the %MDD. The reduction magnitude of the %MDD is dependent on the endpoint and selection of statistical parameters such as the alpha value. Parameters that display effects at a biologically relevant scale will be a better indicator for effects than parameters that are able to detect minor differences that are not biologically relevant.