The retention and mobility of arsenic (As) in soil depend on various physical and chemical factors. The knowledge of the sorption–desorption chemistry of As in soil is necessary for predicting the fate and behavior of As in soil environments. Therefore, this study assessed different organic (sugarcane bagasse and vermicompost) and inorganic amendments (steel slag and fly ash) for their impact on sorption–desorption of As in texturally different contaminated soils (of sandy clay (SC) and sandy clay loam (SCL) texture) to understand the effect of amendments on As retention and mobility. The results showed that the sorption data fitted well with both Langmuir and Freundlich isotherm equations. The As sorption capacity was significantly enhanced with the application of all amendments. At 30 °C, the adsorption maxima (qmax) of SC soils enhanced to a greater extent following the order: steel slag (278 mg kg−1) > sugarcane bagasse (264 mg kg−1) > vermicompost (246 mg kg−1) > fly ash (242 mg kg−1). Whereas, in SCL, the order of qmax was steel slag (145 mg kg−1) > sugarcane bagasse (132 mg kg−1) > fly ash (120 mg kg−1) > vermicompost (118 mg kg−1). Desorption index (DI) was invariably to > 1 at both temperatures with the application of amendments indicating hysteretic desorption of As. The free energy change (ΔG°) was negative in all treatments and soils (indicating a favorable sorption process) with positive entropy change (ΔS°) values. The study recommends steel slag as the most effective amendment for enhancing As (V) retention in contaminated soils, due to its higher sorption capacity compared to other amendments like sugarcane bagasse, vermicompost, and fly ash. The amendments generally improved As sorption in both soils, reducing As mobility and potentially limiting its environmental spread.