With the quick development of up-to-date social informatization, a harm of ubiquitous microwave absorption (MA) has become a significant issue to environment, communication industry, and human health. It is still a challenge to design high-performance absorbing materials to eliminate the harm of MA. In this work, the reduced oxide graphene (rGO) sheets decorated by aluminum nitride (AlN) and spherical-like structures of Fe3O4 nanoparticles were successfully fabricated via a one-pot solvothermal method. The rGO/Fe3O4/AlN hybrid composites as microwave absorbers exhibited superior MA performance by tuning the content ratio of Fe3O4 to AlN and the mass ratio of rGO/Fe3O4/AlN to paraffin was 3:7. Significantly, the minimum reflection loss (RLmin) value of rGO/Fe3O4/AlN-paraffin was − 59.39 dB at 14.24 GHz with a thickness of only 1.8 mm and the effective absorption bandwidth was 5.68 GHz (12.32–18 GHz) when the content ratio of Fe3O4 to AlN was 1:1.5. The introduction of proper content of AlN could enable excellent MA performance of hybrid composites due to the enhancement of the impedance matching and synergistic effect. Therefore, our present work may build up a new strategy for developing prospect MA materials with practical applications by facile preparation routes and low production cost control.