Bypassing Web Application Firewalls Using Deep Reinforcement Learning

Web application firewalls (WAFs) are used for protecting web applications from attacks such as SQL injection, cross-site request forgery, and cross-site scripting. As a result of the growing complexity of web attacks, WAFs need to be tested and updated on a regular basis. There are various tools and techniques to verify the correct performance of WAFs but most of them are manual or use brute-force attacks, so suffer from poor efficacy. In this work, we propose a solution based on Reinforcement Learning (RL) to discover malicious payloads, which can bypass WAFs. We provide an RL framework with an environment compatible with OpenAI gym toolset standards. This environment is employed for training agents to implement WAF circumvention tasks. The agent mutates a malicious payload syntax using a set of modification operators as actions, without changes to its semantic. Then, upon WAF's reaction to the payload, the environment ascertains a reward for the agent. Eventually, based on the rewards, the agent learns a suitable sequence of mutations for any malicious payload. The payloads, which bypass the WAF can determine rules defects, which can be further used in rule tuning for rule-based WAFs. Also, it can enrich the machine learning-based datasets for retraining. We use Q-learning, advantage actor-critic (A2C), and proximal policy optimization (PPO) algorithms with the deep neural network. Our solution is successful in evading signature-based and machine learning-based WAFs. While we focus on SQL injection in this work, the method can be simply extended to use for any string-based injection attacks.