Trade-offs involved in the choice of pot vs field experiments

The ability to control and account for environmental conditions is essential when conducting ecological experiments. Two frequent approaches are pot and field experiments. Pot experiments under common garden conditions are excellent for detecting distinct plant responses to specific experimental treatments. Even maternal effects can be accounted for if plants are grown under the same conditions over generations. Pot experiments are exceptional at detecting even slight differences in response to highly controlled factors, but they are not always a meaningful representation of plant responses under more complex and variable natural habitats.

‘… the most insightful finding by Zhu et al. was that only field mixtures allowed for the identification of traits that were predictive of future species abundance and persistence …’.

Alternatively, field experiments allow for the manipulative control of a limited number of factors under realistic natural conditions. However, most environmental conditions are unaccounted for, making replication difficult, which often results in contradictory evidence. Hence, results may be unquestionably realistic but also unlikely to be consistently replicable, with results being contingent both on location and on time due to factors such as unaccounted variation in soil biota or differences in precipitation among years or between sites.

Plant physiology and ecology often delve into the study of trade-offs. For a scientist designing an experiment, the choice of field vs common garden poses a fundamental choice with its own intrinsic trade-offs. Many individual studies have illustrated the limitations of pot vs field experiments, but a recent study by Zhu et al. (2024; https://doi.org/10.1111/nph.20160), published in New Phytologist, showcases these trade-offs at a particularly large scale (64 different species) and interesting context (exotic vs native species). The authors grew plants of each of the 64 species individually in pots and compared the traits of the different native and exotic species with mixes of the same species planted in the field.

Zhu et al. found interesting results that generally confirmed expectations, but in the field study only, not the one in pots. The study was not without limitations: the comparison of plants grown in pots individually with plants grown in field mixes presented some caveats. It remains unclear whether any difference between native and exotic species - or the absence of a difference - could be due to the absence of competition in the pot experiment or to differences in other factors.

We know that exotic species tend to outperform natives in most scenarios (van Kleunen et al., 2010), but those advantages are sometimes evident only under certain conditions, typically under disturbance (Jauni et al., 2015; Xiao et al., 2016; Montesinos, 2022), when resources are abundant or increasing (Davis et al., 2000; Zhang et al., 2022; Arias et al., 2023) or, alternatively, only when resources are low, or plants experience stress (Funk & Vitousek, 2007; Santamarina et al., 2022). It is possible that the differences observed by Zhu et al. between both experiments were due to the more competitive conditions experienced by plants under field conditions. We cannot discard that a pot experiment involving a mix of species similar to that in the field experiment would not have resulted in similar results. However, at least some previous studies with similar experimental designs found that competition was a factor in reducing - not increasing - trait differences between natives and exotics (Blumenthaal & Hufbauer, 2007). This suggests that the problem might not be the absence of competition in Zhu et al.'s pot experiment, but rather the intrinsic limitations associated with pot experiments.

Perhaps the most insightful finding by Zhu et al. was that only field mixtures allowed for the identification of traits that were predictive of future species abundance and persistence (specific leaf area (SLA), plant height, aboveground biomass). Field mixtures also showed that exotics were overall more persistent than natives, and perennials more than annuals. These results agree with common expectations for native–exotic species interactions, and some previous studies have shown how field studies can detect native and exotic differences that pot experiments are not able to discern (Leffler et al., 2014). Still, previous studies are limited to one or a few species for such comparisons, and meta-analysis such as those by Leffler et al. (2014) provide an integrative view that cannot, however, provide direct evidence in the way that a single study comparing 64 species simultaneously can.

It is therefore comforting to see large scale experiments verifying our theoretical assumptions and yet it is concerning to recognize that pot experiments might be missing so much. Two rapid conclusions can be drawn: first, pot experiments are safe in that they are unlikely to unrealistically magnify or overrepresent existing differences, that is, they are unlikely to result in a type I statistical error (false positive); second, pot experiments might mislead us into ignoring factors and traits that would be highly informative if studied in the field. That is, they are highly likely to result in type II statistical errors (false negative).

This is a sobering insight, and the confirmation of it both by meta-analysis based on numerous, but small, studies (Leffler et al., 2014), and also by large scale experiments such as the one by Zhu et al., should make us reconsider the trade-offs involved. There will always be a need to run experiments in highly controlled pot experiments, but those experiments should focus on specific traits or treatments that have been broadly identified in the field beforehand. Exploring a broad range of traits in pot experiments to determine which ones will be meaningful predictors might appear as an efficient and attractive option, but it could end up being efficient in appearance only. It is likely that such an approach would erroneously lead us to ignore important predictive traits or factors that should have been identified previously via field studies.