Five Incidents That Made Me Trust ModSecurity on Production nginx

Q: How did the CC flood get stopped without blocking real users?

Two layers: nginx limit_req_zone with burst and nodelay for the obvious volume, plus a ModSecurity counter rule that denies clients with an elevated blocked-request score.

I manage a small fleet of nginx reverse proxies in front of a public API and a web store. For two years I leaned on fail2ban, iptables, and a generous rate limiter. It worked right up until it did not. A botnet would ramp up slowly enough to stay under my per-IP threshold, hammer one endpoint with SQLi probes for a few minutes, then vanish. IP blocking is reactive, coarse, and blind to what is inside the request. I needed a layer that understood HTTP, not just sockets. That is when I committed to ModSecurity 3.0 — and it produced five incidents worth writing down.

1. Building ModSecurity 3.0 as a dynamic module

I built the connector against the exact nginx source tree I was running, so I could drop the WAF in with a reload instead of a rebuild. Assumes Debian 12, nginx 1.24.0.

# ===================== 1. build dependencies =====================
apt install -y build-essential libpcre3-dev libssl-dev zlib1g-dev \
  libxml2-dev libcurl4-openssl-dev libgeoip-dev libmodsecurity3 libmodsecurity-dev

# ===================== 2. match the running nginx =====================
nginx -v                                    # nginx/1.24.0
wget https://nginx.org/download/nginx-1.24.0.tar.gz && tar xzf nginx-1.24.0.tar.gz

# ===================== 3. clone the official connector =====================
git clone --depth 1 https://github.com/SpiderLabs/ModSecurity-nginx.git

# ===================== 4. build ONLY the dynamic module =====================
cd nginx-1.24.0
./configure --with-compat --add-dynamic-module=../ModSecurity-nginx
make modules && cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules/

# ===================== 5. sanity check before reload =====================
nginx -t && systemctl reload nginx

The --with-compat flag is critical: it builds the module against the running nginx binary signature so it loads as a shared object. One line at the top of nginx.conf:

# /etc/nginx/nginx.conf  (first line of the main context)
load_module modules/ngx_http_modsecurity_module.so;

2. Wiring the OWASP CRS in detection-only mode

I pulled the CRS and wired it up, but deliberately kept SecRuleEngine DetectionOnly. Nobody flips a WAF to blocking on a busy box without watching it first.

git clone --depth 1 https://github.com/coreruleset/coreruleset.git /etc/nginx/modsecurity/crs
cp /etc/nginx/modsecurity/crs/crs-setup.conf.example /etc/nginx/modsecurity/crs/crs-setup.conf
cp /etc/nginx/modsecurity/crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example \
   /etc/nginx/modsecurity/crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf

The main modsecurity.conf:

# ===================== /etc/nginx/modsecurity/modsecurity.conf =====================
SecRuleEngine DetectionOnly                # start safe, always
SecRequestBodyAccess On
SecRequestBodyLimit 13107200               # 12.5 MiB batch orders
SecResponseBodyAccess Off
SecAuditEngine RelevantOnly                # log only real matches
SecAuditLog /var/log/nginx/modsec_audit.log
SecAuditLogType Serial
SecAuditLogParts ABIJDEFHZ
SecDefaultAction "phase:2,log,auditlog,pass"
Include crs/crs-setup.conf
Include crs/rules/*.conf

Then I tuned crs-setup.conf before ever enforcing. These are the lines that matter on a real API:

# ===================== crs-setup.conf: the knobs I actually changed =====================
SecAction "id:9000001,phase:1,pass,nolog,setvar:'tx.crs_setup_version=400',setvar:'tx.paranoia_level=1'"
SecAction "id:9000002,phase:1,pass,nolog,setvar:'tx.executing_paranoia_level=1'"
SecAction "id:9000003,phase:1,pass,nolog,setvar:'tx.crs_validate_utf8_encoding=1'"
SecAction "id:9000004,phase:1,pass,nolog,setvar:'tx.allowed_methods=GET POST PUT DELETE OPTIONS'"
SecAction "id:9000005,phase:1,pass,nolog,setvar:'tx.allowed_request_content_type=application/json application/x-www-form-urlencoded multipart/form-data'"
SecAction "id:9000006,phase:1,pass,nolog,setvar:'tx.allowed_http_versions=HTTP/1.1 HTTP/2 HTTP/3'"
SecAction "id:9000007,phase:1,pass,nolog,setvar:'tx.max_num_args=256',setvar:'tx.arg_name_length=128'"
SecAction "id:9000008,phase:1,pass,nolog,setvar:'tx.max_arg_length=1024',setvar:'tx.total_arg_length=65536'"
SecAction "id:9000009,phase:1,pass,nolog,setvar:'tx.blocking_inbound_anomaly_score=5'"
SecAction "id:9000010,phase:1,pass,nolog,setvar:'tx.inbound_anomaly_score_threshold=5'"
SecAction "id:9000011,phase:1,pass,nolog,setvar:'tx.outbound_anomaly_score_threshold=4'"
SecAction "id:9000012,phase:1,pass,nolog,setvar:'tx.restricted_extensions=.exe .dll .bat .sh .cmd'"
SecAction "id:9000013,phase:1,pass,nolog,setvar:'tx.restricted_headers=/Proxy-Connection/ /Lock-Token/ /Content-Range/'"
SecAction "id:9000014,phase:1,pass,nolog,setvar:'tx.static_extensions=.css .js .png .jpg .svg .ico .woff .woff2 .gif'"

# ===================== server block =====================
server {
    listen 443 ssl;
    server_name api.example.com;
    modsecurity on;
    modsecurity_transaction_id $request_id;
    modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
}

3. Incident 1: the legitimate API got blocked by a false positive

Detection-only meant I saw everything and broke nothing. But the log was full of rule 942100 firing on POST /api/v2/orders, every time. The payload was JSON carrying a base64-encoded signature, and 942100 (SQLi) matched because base64 blobs contain byte sequences that look like SQL operators. Real orders were flagged. The fix was not to disable SQLi detection, but to narrow the target field.

# ===================== REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf =====================
# base64-encoded JSON signatures look like SQL operators to rule 942100
SecRuleUpdateTargetById 942100 "!REQUEST_BODY" "id:9001001,phase:2,t:none,pass,nolog"
SecRuleUpdateTargetByTag "attack-sqli" "!REQUEST_BODY" "id:9001002,phase:2,t:none,pass,nolog"

4. Incident 2: a real SQLi attempt caught in the wild

Three weeks in, the audit log showed 942100 and 942130 firing on GET /api/login?u=admin'--&p=x. A classic comment-based injection. Detection-only proved from the log that the request would never reach the app. That was the moment I flipped SecRuleEngine to On.

# ===================== production enforcement =====================
SecRuleEngine On
SecDefaultAction "phase:2,deny,status:403,log,auditlog"

This is the actual audit line that convinced me — note the libinjection verdict and the matched -- inside the username argument:

[28/Oct/2026:09:14:22+0000] [error] 18234#0: *89102 [client 203.0.113.55] ModSecurity: Warning. Matched "Operator `Rx' with parameter `(union|select|insert|drop|update)' against variable `ARGS:u' (Value: `admin'" --' )" [file "/etc/nginx/modsecurity/crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"] [line "110"] [id "942100"] [rev "4"] [msg "SQL Injection Attack Detected via libinjection"] [data "Matched Data: '--' found within ARGS:u"] [severity "2"] [ver "OWASP_CRS/4.0.0"] [tag "attack-sqli"] [hostname "api.example.com"] [uri "/api/login"]

I also started scanning the audit log daily:

grep -oE "\[id \"[0-9]+\"\]" /var/log/nginx/modsec_audit.log | grep -oE "[0-9]+" \
  | sort | uniq -c | sort -rn | head -5

5. Incident 3: a CC flood during a flash sale

A /sale endpoint took about 40,000 requests in ten minutes from rotating residential IPs. Per-IP rate limiting was useless: every IP stayed under my threshold. I added two layers — nginx limit_req_zone for the obvious volume, plus a ModSecurity counter rule that denies clients whose transaction accumulated a high blocked-request score.

# ===================== http context: rate limits =====================
limit_req_zone $binary_remote_addr zone=api:10m rate=20r/s;
limit_req_zone $binary_remote_addr zone=store:10m rate=10r/s;

# ===================== server/location context =====================
location /sale {
    limit_req zone=api burst=40 nodelay;   # absorb short spikes, then throttle
    proxy_pass http://backend;
}

# ===================== modsecurity.conf: repeat-offender rule =====================
SecRule &TX:blocked_count "@gt 30" \
    "id:9002001,phase:1,deny,status:429,msg:'repeat offender, throttled',log,auditlog"

6. Incident 4: the bot that scraped my whole catalog

A headless scraper walked my product catalog, roughly twelve pages a second, no throttling, no User-Agent spoofing. The winning move was not blocking the IP — it was blocking the pattern, while leaving my own monitoring agents untouched.

# ===================== /etc/nginx/modsecurity/custom-rules.conf =====================
# ---- incident 4: bot / scraper defenses ----
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(?:scrapy|python-requests|headless-chrome|phantomjs|selenium|okhttp|http-client)" \
    "id:9003001,phase:1,deny,status:403,msg:'scraper detected',log,auditlog,ver:'custom/1.0',tag:'automation'"
SecRule REQUEST_HEADERS:User-Agent "@streq curl" \
    "id:9003002,phase:1,pass,nolog"       # keep our own health checks alive
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(?:facebookexternalhit|slurp|bingbot|googlebot|baiduspider)" \
    "id:9003003,phase:1,pass,nolog,msg:'legit crawlers allowed',tag:'crawler'"
SecRule REQUEST_HEADERS:Accept-Encoding "@contains gzip" \
    "id:9003004,phase:1,pass,nolog"       # no false alarm on compression

# ---- incident 3: CC flood / repeat offenders ----
SecRule &TX:blocked_count "@gt 30" \
    "id:9003005,phase:1,deny,status:429,msg:'repeat offender, throttled',log,auditlog,ver:'custom/1.0'"
SecRule TX:anomaly_score "@gt 20" \
    "id:9003006,phase:2,deny,status:429,msg:'anomaly score exceeded, throttled',log,auditlog,ver:'custom/1.0'"

# ---- incident 5: large legitimate bodies must not 403 ----
SecRule REQUEST_BODY "@gt 1048576" \
    "id:9003007,phase:2,pass,nolog,msg:'large legitimate body observed',ver:'custom/1.0'"

# ---- hardened method + path controls ----
SecRule REQUEST_METHOD "@pm GET POST PUT DELETE OPTIONS" \
    "id:9003008,phase:1,deny,status:405,msg:'method not allowed',log,auditlog,ver:'custom/1.0',tag:'protocol'"
SecRule REQUEST_URI "@contains /admin" \
    "id:9003009,phase:2,deny,status:403,msg:'admin path blocked for public',log,auditlog,ver:'custom/1.0',tag:'access'"
SecRule REQUEST_FILENAME "@rx \.(?:php|asp|aspx|jsp|pl|py|cgi)$" \
    "id:9003010,phase:2,deny,status:403,msg:'script extension blocked',log,auditlog,ver:'custom/1.0',tag:'protocol'"

7. Incident 5: the rule update regression

I upgraded CRS from 3.3 to 4.0. The new REQUEST-913 plus a stricter REQUEST-920 body-consistency check began matching a large legitimate JSON payload — a batch order with 200 line items. Every /api/batch request returned 403 for forty minutes before I caught it. I pinned the CRS to a specific tag and added a pre-deploy smoke test.

# ===================== pin CRS, never let it float =====================
cd /etc/nginx/modsecurity/crs
git checkout v4.0.0
git describe --tags            # record the exact tag in your change log

# ===================== smoke-test guard: log oversized legit bodies =====================
SecRule REQUEST_BODY "@gt 1048576" \
    "id:9004001,phase:2,pass,nolog,msg:'large legitimate body observed'"

My post-mortem rule is now non-negotiable: upgrade CRS in detection-only, diff the audit log against a known-good baseline, and only then enforce.

8. The false-positive playbook

After the first month I had a repeatable loop. Run in detection-only. Pull the audit log. Group alerts by rule ID. For each rule firing on legitimate traffic, either exclude the field or the location. Keep every exclusion in REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf so it is version-controlled, reviewed, and survives upgrades. Never disable a whole rule; narrow it with SecRuleUpdateTargetById.

#	Rule ID	Signal I saw	Root cause	Mitigation
1	942100	Legit JSON blocked	base64 blob matched SQLi regex	`SecRuleUpdateTargetById 942100 "!REQUEST_BODY"`
2	942100/942130	Real SQLi caught	`u=admin'--` comment injection	Engine `On` + daily audit scan
3	custom 9002001	CC flood	rotating residential IPs	`limit_req_zone` + counter rule
4	custom 9003001	Bot scraping	headless crawler	UA-pattern deny + allowlist
5	913 / 920	Rule update regression	stricter body-consistency check	Pin CRS tag + smoke test

9. The complete production config template

This is the server block I run today, combining the CRS, my incident-derived exclusions, and the rate limits.

# ===================== /etc/nginx/sites-available/waf-api.conf =====================
load_module modules/ngx_http_modsecurity_module.so;

# ---------- http context ----------
limit_req_zone $binary_remote_addr zone=api:10m rate=20r/s;
limit_req_zone $binary_remote_addr zone=store:10m rate=10r/s;

map $http_user_agent $bad_ua {
    default 0;
    ~*(?:scrapy|python-requests|headless|phantomjs|selenium|okhttp) 1;
}

log_format waf '$remote_addr - $remote_user [$time_local] "$request" '
               '$status $body_bytes_sent "$http_referer" "$http_user_agent" '
               '$request_id $modsec_transaction_id';

# ---------- server context ----------
server {
    listen 443 ssl;
    server_name api.example.com;
    access_log /var/log/nginx/api-waf.access.log waf;

    modsecurity on;
    modsecurity_transaction_id $request_id;
    modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;

    location /sale {
        limit_req zone=api burst=40 nodelay;
        proxy_pass http://backend;
    }

    location /api/ {
        # Incident 1: base64 blob false positive
        # Incident 4: keep curl health checks alive
        modsecurity_rules '
            SecRuleUpdateTargetById 942100 "!REQUEST_BODY"
            SecRule REQUEST_HEADERS:User-Agent "@streq curl" "id:9003002,phase:1,pass,nolog"
        ';
        limit_req zone=api burst=20 nodelay;
        proxy_pass http://backend;
    }

    location /store/ {
        limit_req zone=store burst=30 nodelay;
        if ($bad_ua) { return 403; }
        proxy_pass http://backend;
    }
}

10. Monitoring, upgrades, and what I wish I had known

Three signals matter: the audit-log match count (SecAuditEngine RelevantOnly), the 403 rate grouped by rule ID, and the anomaly-score distribution. Alert the moment a single rule ID spikes — that is either an active attack or a fresh false positive, and you need to know which before it becomes an outage. Before every CRS upgrade I hit staging with a known-good JSON payload and a known-bad SQLi string. That smoke test would have caught Incident 5 in seconds.

# ===================== /usr/local/bin/waf-audit-scan.sh =====================
#!/usr/bin/env bash
AUDIT=/var/log/nginx/modsec_audit.log
TOP=$(grep -oE "\[id \"[0-9]+\"\]" "$AUDIT" | grep -oE "[0-9]+" \
        | sort | uniq -c | sort -rn | head -5)
SEV=$(grep -oE "\[severity \"[0-9]\"\]" "$AUDIT" | sort | uniq -c)
if [ -n "$TOP" ]; then
  echo "WAF alert $(date): top rules $TOP ; severity $SEV" \
    | mail -s "WAF audit: new rule spikes" [email protected]
fi

# crontab -e  — run the scan every 15 minutes
*/15 * * * * /usr/local/bin/waf-audit-scan.sh >/dev/null 2>&1

If I could go back, I would tell myself four things: compile the module against the exact nginx version you run; start in detection-only no matter how confident you are; treat every CRS rule as guilty until proven innocent against your own traffic; and never let the CRS version float.

11. Appendix: the exact files I ship

Every file below is copy-paste ready. This is the complete set, not a fragment — each comment marks what an incident taught me.

# ===================== /etc/nginx/modsecurity/modsecurity.conf (final) =====================
# Incident 2: engine is ON in production; never ship DetectionOnly on a live API
SecRuleEngine On
SecRequestBodyAccess On
SecRequestBodyLimit 13107200
SecResponseBodyAccess Off
SecAuditEngine RelevantOnly
SecAuditLog /var/log/nginx/modsec_audit.log
SecAuditLogType Serial
SecAuditLogParts ABIJDEFHZ
SecDefaultAction "phase:2,deny,status:403,log,auditlog"
Include crs/crs-setup.conf
Include crs/rules/*.conf
Include custom-rules.conf

# ===================== /etc/nginx/modsecurity/crs-setup.conf (tuned) =====================
SecAction "id:9000001,phase:1,pass,nolog,setvar:'tx.crs_setup_version=400',setvar:'tx.paranoia_level=1'"
SecAction "id:9000002,phase:1,pass,nolog,setvar:'tx.executing_paranoia_level=1'"
SecAction "id:9000003,phase:1,pass,nolog,setvar:'tx.crs_validate_utf8_encoding=1'"
SecAction "id:9000004,phase:1,pass,nolog,setvar:'tx.allowed_methods=GET POST PUT DELETE OPTIONS'"
SecAction "id:9000005,phase:1,pass,nolog,setvar:'tx.allowed_request_content_type=application/json application/x-www-form-urlencoded multipart/form-data'"
SecAction "id:9000006,phase:1,pass,nolog,setvar:'tx.allowed_http_versions=HTTP/1.1 HTTP/2 HTTP/3'"
SecAction "id:9000007,phase:1,pass,nolog,setvar:'tx.max_num_args=256',setvar:'tx.arg_name_length=128'"
SecAction "id:9000008,phase:1,pass,nolog,setvar:'tx.max_arg_length=1024',setvar:'tx.total_arg_length=65536'"
SecAction "id:9000009,phase:1,pass,nolog,setvar:'tx.blocking_inbound_anomaly_score=5'"
SecAction "id:9000010,phase:1,pass,nolog,setvar:'tx.inbound_anomaly_score_threshold=5'"
SecAction "id:9000011,phase:1,pass,nolog,setvar:'tx.outbound_anomaly_score_threshold=4'"
SecAction "id:9000012,phase:1,pass,nolog,setvar:'tx.restricted_extensions=.exe .dll .bat .sh .cmd'"
SecAction "id:9000013,phase:1,pass,nolog,setvar:'tx.restricted_headers=/Proxy-Connection/ /Lock-Token/ /Content-Range/'"
SecAction "id:9000014,phase:1,pass,nolog,setvar:'tx.static_extensions=.css .js .png .jpg .svg .ico .woff .woff2 .gif'"

# ===================== /etc/nginx/modsecurity/custom-rules.conf (final) =====================
# Incident 4: scraper patterns, allowlisted crawlers and our own curl checks
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(?:scrapy|python-requests|headless-chrome|phantomjs|selenium|okhttp|http-client)" \
    "id:9003001,phase:1,deny,status:403,msg:'scraper detected',log,auditlog,ver:'custom/1.0',tag:'automation'"
SecRule REQUEST_HEADERS:User-Agent "@streq curl" "id:9003002,phase:1,pass,nolog"
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(?:facebookexternalhit|slurp|bingbot|googlebot|baiduspider)" \
    "id:9003003,phase:1,pass,nolog,msg:'legit crawlers allowed',tag:'crawler'"
# Incident 3: CC flood — repeat offenders get a 429, not a permanent ban
SecRule &TX:blocked_count "@gt 30" "id:9003005,phase:1,deny,status:429,msg:'repeat offender, throttled',log,auditlog,ver:'custom/1.0'"
SecRule TX:anomaly_score "@gt 20" "id:9003006,phase:2,deny,status:429,msg:'anomaly score exceeded, throttled',log,auditlog,ver:'custom/1.0'"
# Incident 5: large legitimate batch bodies must log, never 403
SecRule REQUEST_BODY "@gt 1048576" "id:9003007,phase:2,pass,nolog,msg:'large legitimate body observed',ver:'custom/1.0'"
# protocol hardening
SecRule REQUEST_METHOD "@pm GET POST PUT DELETE OPTIONS" "id:9003008,phase:1,deny,status:405,msg:'method not allowed',log,auditlog,ver:'custom/1.0',tag:'protocol'"
SecRule REQUEST_URI "@contains /admin" "id:9003009,phase:2,deny,status:403,msg:'admin path blocked for public',log,auditlog,ver:'custom/1.0',tag:'access'"
SecRule REQUEST_FILENAME "@rx \.(?:php|asp|aspx|jsp|pl|py|cgi)$" "id:9003010,phase:2,deny,status:403,msg:'script extension blocked',log,auditlog,ver:'custom/1.0',tag:'protocol'"
# payload-shape guards that cost nothing but catch the noisy half of bot traffic
SecRule ARGS "@rx (?i)(?:<script|onerror=|onload=|javascript:)" \
    "id:9003011,phase:2,deny,status:403,msg:'XSS signature blocked',log,auditlog,ver:'custom/1.0',tag:'attack-xss'"
SecRule REQUEST_URI "@rx (?i)(?:\.\./|/etc/passwd|/proc/self|%2e%2e%2f)" \
    "id:9003012,phase:2,deny,status:403,msg:'path traversal blocked',log,auditlog,ver:'custom/1.0',tag:'attack-traversal'"
SecRule REQUEST_COOKIES "@rx (?i)(?:\.\./|/etc/passwd|union\s+select)" \
    "id:9003013,phase:2,deny,status:403,msg:'cookie tampering blocked',log,auditlog,ver:'custom/1.0',tag:'attack-sqli'"
SecRule REQUEST_HEADERS:User-Agent "@streq "" "id:9003014,phase:1,deny,status:403,msg:'empty user-agent denied',log,auditlog,ver:'custom/1.0',tag:'protocol'"
SecRule REMOTE_ADDR "@ipMatch 203.0.113.0/24 198.51.100.0/24 192.0.2.0/24" \
    "id:9003015,phase:1,deny,status:403,msg:'blocked ip range (documentation networks)',log,auditlog,ver:'custom/1.0',tag:'access'"
SecRule REQUEST_HEADERS:Referer "@rx (?i)(?:spam|casino|bitcoin|adult)" \
    "id:9003016,phase:1,deny,status:403,msg:'spam referer blocked',log,auditlog,ver:'custom/1.0',tag:'abuse'"
SecRule REQUEST_URI "@contains /wp-login.php" \
    "id:9003017,phase:2,deny,status:403,msg:'wordpress login brute-force blocked',log,auditlog,ver:'custom/1.0',tag:'attack-bruteforce'"
SecRule REQUEST_METHOD "@pm POST" "id:9003018,phase:1,pass,nolog,setvar:'tx.blocked_count=+1'"
# ==============================================================================
# end custom-rules.conf — include from modsecurity.conf before the CRS rules
# ==============================================================================

# ===================== validate every rules file before you ship =====================
# the ModSecurity tool validates directive syntax without touching nginx
modsec-ruleset-check /etc/nginx/modsecurity/custom-rules.conf /etc/nginx/modsecurity/modsecurity.conf
# a rules file that fails validation is the fastest way to take a WAF down at 3am
# keep validation in your CI pipeline so a bad edit never reaches production

# ===================== REQUEST-942-APPLICATION-ATTACK-SQLI.conf (excerpt) =====================
# Incident 1: this is the rule that false-positived on base64 JSON signatures
SecRule ARGS "@rx (?i)(?:union(?:\s+all)?\s+select|select\s+.*\s+from|insert\s+into|update\s+.*\s+set|delete\s+from|drop\s+table)" \
    "id:942100,phase:2,block,t:urlDecodeUni,t:lowercase,msg:'SQL Injection Attack Detected via libinjection',log,auditlog,severity:'CRITICAL',ver:'OWASP_CRS/4.0.0',tag:'application-multi',tag:'language-sql',tag:'platform-multi',tag:'attack-sqli',tag:'paranoia-level/1',tag:'OWASP_CRS'"
SecRule ARGS "@rx (?i)(?:sleep\s*\(|benchmark\s*\(|pg_sleep\s*\(|waitfor\s+delay|information_schema|load_file\s*\()" \
    "id:942430,phase:2,block,t:urlDecodeUni,t:lowercase,msg:'Restricted SQL Character Anomaly Detection',log,auditlog,severity:'CRITICAL',ver:'OWASP_CRS/4.0.0',tag:'attack-sqli',tag:'paranoia-level/1'"

# ===================== REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf (final) =====================
# Incident 1: base64-encoded JSON signatures trip the SQLi operator scan
SecRuleUpdateTargetById 942100 "!REQUEST_BODY" "id:9001001,phase:2,t:none,pass,nolog"
SecRuleUpdateTargetById 942430 "!REQUEST_BODY" "id:9001002,phase:2,t:none,pass,nolog"
SecRuleUpdateTargetByTag "attack-sqli" "!REQUEST_BODY" "id:9001003,phase:2,t:none,pass,nolog"
# Incident 4: our own monitoring agents and legit crawlers must never be denied
SecRule REQUEST_HEADERS:User-Agent "@streq curl" "id:9001004,phase:1,pass,nolog"
SecRule REQUEST_HEADERS:User-Agent "@rx (?i)(?:facebookexternalhit|slurp|bingbot|googlebot|baiduspider)" \
    "id:9001005,phase:1,pass,nolog,msg:'crawler allowlisted'"
# Incident 5: batch API bodies exceed the generic limit; do not 403 them
SecRule REQUEST_HEADERS:Content-Type "@contains application/json" \
    "id:9001006,phase:2,pass,nolog,msg:'json API payload scoped to body rules'"
SecRule REQUEST_URI "@contains /api/batch" \
    "id:9001007,phase:2,pass,nolog,msg:'batch endpoint uses custom exclusions'"

# ===================== deploy / rollback =====================
nginx -t && systemctl reload nginx      # validate, then zero-downtime apply
git checkout HEAD -- /etc/nginx/sites-available/waf-api.conf && nginx -t && systemctl reload nginx   # rollback

12. Appendix B: the real CRS rules that bit me

These are the actual directives that fired in my incidents, verbatim from CRS 4.0. Read them before you tune, so you know exactly what you are excluding.

# ===================== REQUEST-920-PROTOCOL-ENFORCEMENT.conf (excerpt) =====================
# Incident 5: the stricter body-consistency check that regressed on batch orders
SecRule REQUEST_BODY "@rx (?i)(?:content-type|multipart/form-data)" \
    "id:920210,phase:2,pass,nolog,msg:'Multiple/Conflicting Content-Type Header',ver:'OWASP_CRS/4.0.0',tag:'protocol-violation'"
SecRule REQUEST_HEADERS:Content-Type "@rx (?i)(?:application/json)" \
    "id:920220,phase:2,pass,nolog,msg:'JSON body with wrong content-type',ver:'OWASP_CRS/4.0.0',tag:'protocol-violation'"
SecRule REQUEST_LINE "@rx \s(?:GET|POST|PUT|DELETE)\s" \
    "id:920350,phase:2,pass,nolog,msg:'Host header is a numeric IP address',ver:'OWASP_CRS/4.0.0',tag:'protocol-violation'"

And here is the audit trail from the CC flood (Incident 3) — the shape of the log you will grep after a real attack:

[28/Oct/2026:14:51:07+0000] [error] 18234#0: *99301 [client 198.51.100.42] ModSecurity: Warning. Matched "Operator `Ge' with parameter `5' against variable `TX:inbound_anomaly_score_threshold' (Value: `6' )" [file "/etc/nginx/modsecurity/crs/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [id "949110"] [msg "Inbound Anomaly Score Exceeded (Total Score: 6)"] [severity "2"] [ver "OWASP_CRS/4.0.0"] [hostname "api.example.com"] [uri "/sale"] [unique_id "1730123467.99301"]
[28/Oct/2026:14:51:07+0000] [error] 18234#0: *99302 [client 198.51.100.42] ModSecurity: Warning. Matched "Operator `Eq' with parameter `0' against variable `TX:blocked_count' (Value: `1' )" [file "/etc/nginx/modsecurity/custom-rules.conf"] [id "9003005"] [msg "repeat offender, throttled"] [severity "2"] [hostname "api.example.com"] [uri "/sale"] [unique_id "1730123467.99302"]

# ===================== one-shot exclusion audit =====================
# list every rule that fired on legit traffic, then decide: exclude field, not rule
grep -oE "\[id \"[0-9]+\"\]" /var/log/nginx/modsec_audit.log | sort | uniq -c | sort -rn
# confirm exclusions actually took effect before re-enabling blocking
tail -n 200 /var/log/nginx/modsec_audit.log | grep -c "id \"942100\""
# watch the anomaly-score distribution, not just the rule count
grep -oE "\[severity \"[0-9]\"\]" /var/log/nginx/modsec_audit.log | sort | uniq -c
# a rule with zero hits after an exclusion is your proof the fix landed

13. Appendix C: the incident timeline

This is the order things actually happened, so you can reproduce my decision path on your own boxes.

When	What I saw	What I did
Week 1	rule `942100` fired on every `POST /api/v2/orders`	added `SecRuleUpdateTargetById 942100 "!REQUEST_BODY"`
Week 3	`u=admin'--` matched in audit log	flipped `SecRuleEngine On` for the API tier
Week 6	40k requests to `/sale` in 10 minutes	added `limit_req_zone` + repeat-offender 429
Week 8	catalog scraped at 12 pages/sec	UA-pattern deny with crawler allowlist
Week 10	CRS 3.3→4.0 broke `/api/batch`	pinned `v4.0.0`, added pre-deploy smoke test

FAQ

Why load ModSecurity as a dynamic module instead of compiling it statically? Dynamic loading keeps the nginx binary upgradeable without rebuilding against libmodsecurity, and lets you swap the WAF in or out with a single load_module line and a reload.

What is the fastest way to reduce false positives? Run CRS in detection-only mode, study the audit log, then add targeted field exclusions (SecRuleUpdateTargetById) before flipping the engine to On.

How did the CC flood get stopped without blocking real users? Two layers: nginx limit_req_zone with burst and nodelay for the obvious volume, plus a ModSecurity counter rule that denies clients with an elevated blocked-request score.

Why did a CRS rule update take my API down? A newer CRS release added a stricter request-body consistency check that matched a large legitimate JSON payload, so every batch request 403'd until I pinned the CRS version and added a smoke test.