Several weeks ago one of the servers I manage ended up with a lot of comment spam. After several hours of searching through the logs and correlating the IP’s, I found out that a majority of the hosts were listed in the BlockList.de’s bots.txt file. I didn’t want to import ALL the IP’s on the list as I noticed that quite a few could be pushed into fewer lines using CIDR notation.
I set out to compress this and ended up with this Gist that can do exactly that.
$botList = new BlocklistDe(‘bots.txt’);
This particular server was using Ubuntu, so I pushed the 13k addresses into the iptables configuration. Spam has been cut quite dramatically.