Consistency in marine heatwave experiments for ecological relevance and application: Key problems and solutions

Extreme events attributed to climate change are increasingly being recognized for their potentially devastating effects on species and ecosystems (Harris et al. 2018). The occurrence and intensities of marine heatwaves (MHWs) are increasing and so will their impact on marine ecosystems (Holbrook et al. 2020). Determining their ecological impact on coastal and pelagic ecosystems remains a major component of climate change research (Harvey et al. 2022; Hemraj et al. 2023), and, therefore, the number of studies examining the impact of MHWs on species, biodiversity, and ecosystem processes is increasing yearly (Fig. 1a,b). Among these studies are purely experimental work that aims to determine the impact of different intensities or frequencies of MHWs on an organism's genetics, physiology, behavior, or community interactions. Such experimental work remains fundamental to increasing our understanding of specific effects of MHWs that either cannot be measured in situ or are integral in estimating possible future impacts of MHWs on organisms. Given the large inference often based on these studies, it is essential that they follow a consistent experimental protocol that is representative of MHWs dynamics within different geographical regions and thus provide ecologically relevant information within the regional context. Nonetheless, there are lingering inconsistencies among experimental designs that make it difficult to compare the ecological outcomes of experimental studies and, in the worst cases, some are not representative of MHWs at all. Among these inconsistencies are issues with (1) the duration of the experimental study, (2) the choice of baseline temperature (control) to which MHW impacts are compared, and (3) the choice of intensity treatments.

The generally accepted definition of a MHW is anomalous seawater temperature above the 90^th percentile of long-term climatology that lasts a minimum of 5 days (Hobday et al. 2018). Many experiments have been carried out for shorter durations, which by definition represent “heat spikes,” yet results were discussed in the MHW context. In addition, in several instances, experiments have used control temperatures that are, for example, a mean annual average. These neglect natural variability in temperature and the effect of temperature variation on thermal physiology across multiple time scales (from daily to seasonal). Finally, multiple studies have used MHW intensities (temperature anomaly above 90^th percentile of climatology) based on expected global mean intensities. These do not convey regionally relevant information on the impact of MHWs but rather a general estimation of change in thermal physiology in relation to temperature increase. Here, we reviewed the experimental studies examining the impact of MHWs on organisms from the last 5 yr (2019–2023) to highlight the extent of inconsistency in MHW simulation parameters used and provide key solutions that will help avoid these inconsistencies in future. Implementing these principles will render inferences made from experimental work on MHWs more comparable and ecologically relevant, increasing their applicability toward population and ecosystem management.

We examined studies specifically designed to test the effect of MHWs on organisms to highlight the problems with experimental designs. The overall aim of this analysis is to promote consistent good practice within experimental research on MHWs so that studies identify ecologically relevant effects of MHWs on organisms and within different regions. This in turn promotes comparative studies regionally and globally, and conservation or management applications which are regionally relevant. To classify studies, we used the broad search term “(marine heatwave) OR (marine heat wave)” in the Web of Science from 01 July 2018, after Hobday et al. (2018) published the definition of MHWs, to 01 December 2023 and downloaded all 2508 studies that were published within that timeframe. The resulting list of studies purported to be either direct experimental tests of MHWs or discussed their results within the MHW context. We screened the papers to identify those that were purely experimental and specifically aimed at identifying the effect of MHW conditions on marine organisms (n = 169; Supplementary 1). We then examined the experimental designs of these purely experimental studies using the classification scheme for MHWs developed by Hobday et al. (2016, 2018) and classified them as either “no issues,” “issues with experimental duration,” “issues with choice of baseline (control),” “issues with intensity,” “having at least two of these issues,” or “having all three of these issues.”

In these last 5 yr (2019–2023), there was an increase in studies on the impact of MHWs on organisms (Fig. 1a). Specifically, there were 11, 24, 33, 41, and 55 purely experimental studies in 2019, 2020, 2021, 2022, and 2023, respectively (Fig. 1b). These studies were composed of 73% examining single species response and 27% examining multiple species or community responses (Fig. 1c). Importantly, only ~ 55% of these studies had no problems with experimental design. About two-thirds of the studies that used relevant experimental MHW conditions directly used a 30-yr climatology to infer MHW conditions (as suggested by Hobday et al. 2018) or based their MHW conditions on other papers that used a 30-yr climatology to model MHWs within similar a region of study. The remaining studies based their experimental MHW conditions on field measurements of previous MHWs. Critically, 16% of studies used experimental durations that were not consistent with the definition of MHWs (Fig. 1d), that is, less than 5 d. In addition, 24% of studies used an inappropriate baseline temperature (control) and 33% used MHW intensities that were not locally relevant (e.g., based on observations or projections for other regions) (Fig. 1d). Finally, 24% and 8% of studies contained at least two design issues or all three issues, respectively (Fig. 1d). We calculated the percentage of studies containing each of the flaws for each year. We found that there were no particular trends in the percentage of studies containing each of the flaws over the years, except for the number of studies using the wrong experimental duration (< 5 d), which increased from about 9% in 2019 and 2020 to about 23% in 2023.

The most fundamental problem with many recent experimental designs is the duration of experiments. By definition, a MHW is represented by a temperature anomaly that lasts 5 d or more above the 90^th percentile of the long-term climatology (Hobday et al. 2016). This current standard was specifically identified from sensitivity analyses performed using high-resolution (1/4°), global, daily SSTs from the AVHRR satellite data (Hobday et al. 2016) to increase consistency in MHW identification. Anomalies that last less than 5 d are classified as heat spikes unless they are followed by 5 or more anomalous warm days within less than 2 d. Thus, durations of less than 5 d should not be used in experimental designs to investigate MHW impacts and should rather be presented and discussed within an acute heat spike or thermal stress context rather than MHWs. Indeed, the differentiation between durations is important to integrate longer effects of thermal stress on organisms. Most organisms possess a capacity for short-term physiological stress tolerance because of natural variation in their environments (Seebacher et al. 2015; Kroeker et al. 2020). MHWs, on the other hand, are novel conditions that can stress organisms beyond their short-term plasticity (Hemraj et al. 2020) and unless this is tested for in experiments the resulting inferences may be misleading.

Second, the baseline temperature (control) used should be ecologically relevant to the local environment. Depending on the hypotheses being tested, using the mean maximum (across years) summer (or winter) temperature, or water temperature at which an organism is collected prior to the experiment, are appropriate as they are most representative of a natural thermal and physiological state at which the organisms are likely to experience a MHW. On the other hand, the use of a temperature such as mean yearly temperature is inappropriate as it does not consider changes in the thermal physiology of organisms across seasons (Bradshaw and Holzapfel 2010) and regards organisms as having a constant physiological state. In addition, adding temperature intensities (e.g., +4°C) to a mean yearly temperature might not be representative of a locally relevant MHW and not test physiologically meaningful questions, especially in temperate regions where seasonal temperature variation is high.

Finally, the choice of intensity of temperature anomaly is extremely important. Inappropriate choice of experimental heatwave intensity, either excessively high or low, may convey no biological relevance. Regional drivers of MHW and their intensities vary immensely (Holbrook et al. 2019). Therefore, unless local ecologically relevant MHW intensities are used in experiments, these are unlikely to uncover useful information on the response of organisms to MHW within the region being studied. Therefore, such experiments do not provide a relevant assessment of the potential impacts of MHWs, and thus also cannot be used to inform ecosystem management or conservation measures. Identification of an ecologically relevant MHW intensity can either be done by using the intensity of a recent MHW within the local region, estimating the current intensities of MHWs within the region from local daily sea surface temperature data (e.g., using tools such as HeatwaveR; Schlegel and Smit 2018), or modeling future MHW intensities for the region based on CMIP6 models. In addition, reporting the category of the MHW being tested (moderate, strong, severe, extreme; Hobday et al. 2018) can provide a more consistent assessment of impacts across studies because they provide a standardized level of intensity above the local conditions.

Considering the rate at which experimental studies on the impact of MHWs on organisms and ecosystems is increasing annually, it is imperative that experimental designs are consistent and, more importantly, biologically relevant. This is especially important to generate comparative studies with regional, global, and biological applications such as ecosystem management and conservation. While the studies with key problems in experimental design remain informative of the physiological response of organisms to thermal stress, they do not convey comparable information on current or future MHW impacts. Therefore, to enhance the broader relevance of such important experimental work, we suggest that experimental designs should focus on (1) using locally relevant experimental duration consistent with current or projected MHWs, (2) relevant baseline temperatures that represent natural settings where an organism is likely to be exposed to a MHW (either now or into the future), and (3) carefully selected intensity and category-based local heatwave dynamics. Following these guidelines and implementing these three fundamental principles in experimental designs will help with determining and understanding the heterogeneity in the ecological impact of MHWs locally and globally, which, in turn, will help with better management of ecosystems.