This paper comprehensively studies the interface characters and behaviors of asphalt-aggregate by laboratory tests and molecular dynamics (MD) simulations. To accurately build the molecular model of asphalt-aggregate interface system and explore the nano-scale adhesion mechanism, the physicochemical composition of SARA components (saturate, aromatic, resin, and asphaltene) is characterized by macro–micro tests including SARA components separation and elemental analyzer. The mineral composition of aggregate is analyzed by X-ray diffraction test. The adhesion work, diffusion behavior and relative distribution of SARA components on aggregate surface are analyzed based on MD simulations. The results show that the influence of aggregate type on adhesion work is higher than that of asphalt type. The interfacial adhesion strength law obtained by pull-off test is highly consistent with the simulation results. Diffusion behavior is related to the polarity and proportion of SARA components and mineral types. Polar components have strong adhesion to minerals and are concentrated near the surface of minerals, and they are easily adsorbed on the surfaces of alkaline minerals such as calcite and albite. The adsorption characteristics of mineral surface will be affected by the proportion of SARA components.