BALG: Bypassing Application Layer Gateways using multi-staged encrypted shellcodes

Abstract

Modern attacks are using sophisticated and innovative techniques. The utilization of cryptography, self-modified code, and integrated attack frameworks provide more possibilities to circumvent most existing perimeter security approaches, such as firewalls and IDS. Even Application Layer Gateways (ALG) which enforce the most restrictive network access can be exploited by using advanced attack techniques. In this paper, we propose a new attack for circumventing ALGs. By using polymorphic and encrypted shellcode, multiple shellcode stages, protocol compliant and encrypted shell tunneling, and reverse channel discovery techniques, we are able to effectively bypass ALGs. The proposed attack consists of four phases with certain requirements and results. We implemented the initial shellcode as well as the different stages and conducted the practical attack using an existing ALG. The possibility to prevent this attack with existing approaches is discussed and further research in the area of perimeter security and log management is motivated.