Description
Hello.
I have compiled nginx 1.13.7 + modsecurity 3.0 + https://github.com/SpiderLabs/ModSecurity/pull/1667/files patch
modsecurity is enabled globally in the http section for all virtual hosts
If modsecurity is enabled and number of vhosts are increased nginx workers are starting to consume more memory.
I can see a clear relationship between the number of modsecurity rules, the number of vhosts and the memory consumption
If I have a lot of vhosts and a lot of modsec rules memory usage is high
If I decrease either the number of rules or the number of virtual hosts memory footprint starts to drop
It appears to me that instead of storing all modsec rules on a central place in the memory and doing lookups/searches in there, you are allocating separate modsecurity structures that include all rules per server/virtual host. Not sure.
See the table below:
version - nginx version
vhosts - number of vhosts
modsec - number of modsec rules
virt - virtual memory
rss - rss memory consumption
mem/vhost - rough calculation rss / vhosts
rules_mem - rough calculation rss (modsec on) - rss (modsec off)
mem/rule - rough calculation rss / rules
From that it appears to me that memory is growing faster depending on the number of virtual hosts not on the number of rules we have. Not sure why is that when modsec is included globally into the http section.
version vhosts modsec rules virt rss mem/vhost rules_mem ~rough mem/rule
1.10.3 14 no 0 141m 19.51m 1.39m 0 0
1.10.3 1779 no 0 823m 702m 0.39m 0 0
1.10.3 1779 no 0 823m 703m 0.39m 0 0
1.13.7 1779 no 0 854m 732m 0.41m 0 0
1.13.7 1779 off 0 1007m 814m 0.45m 0 0
1.13.7 1779 off 0 1269m 1077m 0.60m 0 0
1.13.7 1779 off 0 1269m 1077m 0.60m 0 0
1.13.7 1779 off 0 1277m 1126m 0.63m 0 0
1.13.7 1779 on 0 1277m 1126m 0.63m 0 0
1.13.7 1779 on 1661 2541m 2342m 1.31m 1216m 0.73m
1.13.7 1779 on 3288 3306m 3063m 1.72m 2249m 0.68m
1.13.7 1779 on 3288 3798m 3575m 2.00m 2449m 0.74m
1.13.7 891 on 3288 2021m 1768m 1.98m 642m 0.19m
I hope this helps.