Hölder network for improved adversarial robustness

A small Lipschitz constant can help improve robustness and generalization by restricting the sensitivity of the model to input perturbations. However, overly aggressive constraints may also limit the network’s ability to approximate complex functions. In this paper, we propose the Hölder network, a novel architecture utilizing $\alpha$-rectified power units ($\alpha$-RePU). This framework generalizes Lipschitz-constrained networks by enforcing $\alpha$-Hölder continuity. We theoretically prove that $\alpha$-RePU networks are universal approximators of Hölder continuous functions, thereby offering greater flexibility than models with hard Lipschitz constraints. Empirical results show that the Hölder network achieves comparable accuracy and superior adversarial robustness against a wide range of attacks (e.g., PGD and $l_{\infty }$) on both image classification and tabular data benchmarks.