More info on the Doyensec blog
Many default regular expression parsers have unbounded worst-case complexity. Regex matching may be quick when presented with a matching input string. However, certain non-matching input strings can make the regular expression matcher go into crazy backtracking loops and take ages to process. This can cause denial of service, as the CPU will be stuck trying to match the regex.
This tool is designed to:
- find regular expressions which are vulnerable to ReDoS
- give an example malicious string which will cause catastrophic backtracking
This reflects the complexity of the regular expression matcher's backtracking procedure with respect to the length of the entered string.
Cubic complexity here means that if the vulnerable part of the string is doubled in length, the execution time should be about 8 times longer (2^3). For exponential ReDoS with starred stars e.g.
(a*)*$ a fudge factor is used and the complexity will be greater than 10.
For explotability, cubic complexity or higher is typically required unless truly giant strings are allowed as input.
regexploit and enter the regular expression
v\w*_\w*_\w*$ at the command line.
$ regexploitv\w*_\w*_\w*$Pattern: v\w*_\w*_\w*$---Worst-case complexity: 3 ⭐⭐⭐ (cubic)Repeated character: [5f:_]Final character to cause backtracking: [^WORD]Example: 'v' + '_' * 3456 + '!'
\w*_\w*_\w* contains three overlapping repeating groups (\w matches letters, digits and underscores). As showed in the line
Repeated character: [5f:_], a long string of
_ (0x5f) will match this section in many different ways. The worst-case complexity is 3 as there are 3 infinitely repeating groups. An example to cause ReDoS is given: it consists of the required prefix
v, a long string of
_ and then a
! (non-word character) to cause backtracking. Not all ReDoSes require a particular character at the end, but in this case, a long string of
_ will match the regex successfully and won't backtrack. The line
Final character to cause backtracking: [^WORD] shows that a non-matching character (not a word character) is required at the end to prevent matching and cause ReDoS.
As another example, install a module version vulnerable to ReDoS such as
pip install ua-parser==0.9.0. To scan the installed python modules run
Importing ua_parser.user_agent_parserVulnerable regex in /somewhere/.env/lib/python3.9/site-packages/ua_parser/user_agent_parser.py #183Pattern: \bSmartWatch *\( *([^;]+) *; *([^;]+) *;Context: self.user_agent_re = re.compile(self.pattern)---Worst-case complexity: 3 ⭐⭐⭐Repeated character: Example: 'SmartWatch(' + ' ' * 3456Worst-case complexity: 3 ⭐⭐⭐Repeated character: Example: 'SmartWatch(0;' + ' ' * 3456Vulnerable regex in /somewhere/.env/lib/python3.9/site-packages/ua_parser/user_agent_parser.py #183Pattern: ; *([^;/]+) Build[/ ]Huawei(MT1-U06|[A-Z]+\d+[^\);]+)[^\);]*\)Context: self.user_agent_re = re.compile(self.pattern)---Worst-case complexity: 3 ⭐⭐⭐Repeated character: [[0-9]]Example: ';0 Build/HuaweiA' + '0' * 3456...
For each vulnerable regular expression it prints one or more malicious string to trigger ReDoS. Setting your user agent to
;0 Build/HuaweiA000000000000000... and browsing a website using an old version of ua-parser may cause the server to take a long time to process your request, probably ending in status 502.
Optionally make a virtual environment
python3 -m venv .envsource .env/bin/activate
Now actually install with pip
pip install regexploit
Regexploit with a list of regexes
Enter regular expressions via stdin (one per line) into
or via a file
cat myregexes.txt | regexploit
Extract regexes automatically
Parses Python code (without executing it) via the AST to find regexes. The regexes are then analysed for ReDoS.
regexploit-py my-project/regexploit-py "my-project/**/*.py" --glob
This will use the bundled NodeJS package in
Those regexes are fed into the python ReDoS finder.
regexploit-js my-module/my-file.js another/file.js some/folder/regexploit-js "my-project/node_modules/**/*.js" --glob
Search for regexes in all the python modules currently installed in your path / env. This means you can
pip install whatever modules you are interested in and they will be analysed. Cpython code is included.
N.B. this doesn't parse the python code to an AST and will only find regexes compiled automatically on module import. Modules are actually imported, so code in the modules will be executed. This is helpful for finding regexes which are built up from smaller strings on load e.g. CVE-2021-25292 in Pillow
JSON / YAML
Yaml support requires pyyaml, which can be installed with
pip install regexploit[yaml].
regexploit-json *.jsonregexploit-yaml *.yaml
- CVE-2020-5243: uap-core affecting uap-python, uap-ruby, etc. (User-Agent header parsing)
- CVE-2020-8492: cpython's urllib.request (WWW-Authenticate header parsing)
- CVE-2021-21236: CairoSVG (SVG parsing)
- CVE-2021-21240: httplib2 (WWW-Authenticate header parsing)
- CVE-2021-25292: python-pillow (PDF parsing)
- CVE-2021-26813: python-markdown2 (Markdown parsing)
- CVE-2021-27290: npm/ssri (SRI parsing)
- CVE-2021-27291: pygments lexers for ADL, CADL, Ceylon, Evoque, Factor, Logos, Matlab, Octave, ODIN, Scilab & Varnish VCL (Syntax highlighting)
- CVE-2021-27292: ua-parser-js (User-Agent header parsing)
- CVE-2021-27293: RestSharp (JSON deserialisation in a .NET C# package)
- bpo-38804: cpython's http.cookiejar (Set-Cookie header parsing)
- SimpleCrawler (archived) (HTML parsing)
- CVE-2021-28092: is-svg (SVG parsing)
- nuget.org, NuGetGallery and NuGet.Client (Parsing NuGet package IDs)
- markdown (python) (Markdown parsing)
- ansi-html (nodejs) (ANSI parsing)
- Plus unpublished bugs in a handful of pypi, npm, ruby and nuget packages
This tool has been created by Ben Caller of Doyensec LLC during research time.