Home
CSF/LFD, Fail2Ban and Cloudflare - an UnHoly Trinity
- Details
- Written by: J Grant Forrest
- Category: Software and Coding
- Hits: 23
Whilst doing some housekeeping, I saw a large number of entries in my Apache error_log of the form:
AH00124: Request exceeded the limit of 5 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.
If you ask google about AH00124, you get a lot of results that point to the problem being recursive redirects in .htaccess files, especially with Wordpress sites. As the error suggests, you can set the Apache LogLevel to Debug but I didn't find this very helpful. I posted in the cPanel Security forum as the debug entries suggested the problem may be one of the *.shtml files that cPanel uses but that was as far as I could get.
With no obvious fix, the next option was to monitor and block the offending IPs. I looked at Fail2Ban. If you haven't heard of it, it's an open source tool that allows you to scan and monitor log files. You create 'Jails' that are essentially actions taken when a match is found, such as blocking an IP address in your firewall.
I use the Configserver firewall that was recently rescued by WHM/cPanel, having been abandoned by its original authors. There's debate about whether CSF/LFD can play nicely with Fail2Ban but some report that the two can co-exist if you're careful to avoid conflicts.
My aim was simple - scan the Apache error_log to match a couple of strings, get the origin IP and use CSF to ban it in the iptables firewall. I don't use firewalld so can't comment on integration with that.
I noticed that my existing CSF firewall rules weren't blocking anything, in spite of matching the IPs in the log. This was because I use Cloudflare as a proxy. Apache could get the origin IP because I have the mod_remoteip module but the firewall saw everything coming in on ports 80 and 443 with Cloudflare IPs which were in the allowlist.
Two strategies were possible - either I tried to block the attacking IPs using Cloudflare's WAF or I abandon the Cloudflare proxy and use my CSF firewall.
The difficulty with using Cloudflare is (a) it's expensive and (b) there's a limit (1,000 ?) to the number of IPs you can block before triggering further expense. Although CSF (and fail2ban) have tinkered with interfaces to the Cloudflare API, not having a big budget meant that I decided to turn off the proxy and look at integration between CSF and Fail2Ban.
As it turned out, this was easier than I thought.
I created two new jails in fail2ban:
apache-redirect-failapache-primary-script-unknown
and enabled the existing apache-nohome jail.
The two important elements for each jail are the filter and the action.
The filter is essentially a regex to match a string in the log file and the action is what happens when a match is found.
In this case, the filters were very simple:
failregex = ^%(_apache_error_client)s (AH00124: )?Request exceeded the limit of 5 internal redirects due to probable configuration errorfailregex = ^%(_apache_error_client)s (AH01071: )?Got error 'Primary script unknown'The action was also pretty simple. I created a new one called csf-ip-deny with the following:
actionban = csf --deny <ip> "banned by Fail2Ban <name>"; csf -rcsf.deny file and restarts LFD. It's possible to unban too but I have my csf.deny file capped at 1,000 so I didn't try to over-complicate matters by managing the bantime.