Bifurcation analysis and Turing pattern formation on multiplex networks of ecological model with trade-off dynamics

Ecological trade-offs occur when organisms balance conflicting demands, such as survival and reproduction. Predators may risk injury or death when pursuing dangerous prey, affecting their survival. Similarly, prey must balance reproductive efforts against safety, while predators must weigh food acquisition against the risk of harm. In order to represent prey collective defense, we present a ecological model in this study that includes prey growth affected by fear and a Monod-Haldane functional response. We explore how group defense influences prey trade-off dynamics and introduce a predator death function to represent losses due to dangerous prey encounters. The model reveals complex dynamics, including bistability and several bifurcations: transcritical, saddle-node, Hopf, Bogdanov–Takens, Bautin, homoclinic, and limit cycle bifurcations. Our results indicate that high amount of group defense can lead to predator extinction, highlighting the trade-off between food acquisition and safety. We further extend the model to include cross-diffusion, accounting for species-specific movement patterns. We theoretically derive Turing conditions for both continuous and multiplex domains, demonstrating how cross-diffusion and network topology can destabilize homogeneous steady states, leading to diverse Turing patterns. Our findings reveal that over time prey can drive predator extinction, emphasizing the trade-off between food acquisition and safety in predator–prey systems. Additionally, we observe that variations in the average degree of the Lattice can significantly influence the emergence of Turing patterns and the spatial distribution of species.