Spatiotemporal and Trade-Off Dynamics in Prey–Predator Model with Domed Functional Response and Fear Effect

In the ecological scenario, predators often risk their lives pursuing dangerous prey, potentially reducing their chances of survival due to injuries. Prey, on the other hand, try to strike a balance between reproduction rates and safety. In our study, we introduce a two-dimensional prey–predator model inspired by Tostowaryk’s work, specifically focusing on the domed-shaped functional response observed in interactions between pentatomid predators and neo-diprionid sawfly larvae. To account for the varying effectiveness of larval group defense, we incorporate a new component into the response equation. Our investigation delves into predator trade-off dynamics by adjusting the predator’s mortality rate to reflect losses incurred during encounters with dangerous prey and prey’s trade-off between safety and reproduction rate incorporating this domed-shaped functional response. Our model demonstrates bistability and undergoes various bifurcations, including transcritical, saddle-node, Hopf, Bogdanov–Takens, and Homoclinic bifurcations. Critical parameters impact both predator and prey populations, potentially leading to predator extinction if losses due to dangerous prey encounters become excessive, highlighting the risks predators face for their survival. Furthermore, the efficacy of group defense mechanisms can further endanger predators. Expanding our analysis to a spatially extended model under different perturbations, we explore Turing instability to explain the relationship between diffusion and encounter parameters through both stationary and dynamic pattern formation. Sensitivity to initial conditions uncovers spatiotemporal chaos. These findings provide valuable insights into comprehending the intricate dynamics of prey–predator interactions within ecological systems.