Accuracy of the different sources RIPE/IPdeny/MaxMind. I was surprised...

[QuaxEros]

Hi @friendly-bits,
When i was researching an issue with a certain IP address which from the # whois <IP> command showed it to be in a Seychelles registered block i found that MaxMind was the only source not showing this correctly.
While this CC (SC) is not whitelisted i received massive spam from this IP on my mail server. So after the whois command i used the check-ip-in-source.sh script and queried the 3 sources. RIPE & IPdeny gave a positive response but MaxMind said it wasn't in the SC block.

• The script doesn't source the geoip-shell.conf file for the MaxMind licence & key. I also have the MaxMind client installed on my servers so /etc/GeoIP.conf could have been sourced as well.

I went on to the MaxMind site and saw that they only update the databases on Tuesday and Friday 12:00 CET (i believe), but they state a 99.8% accuracy (they seem quite US minded though). The API should be more accurate and updated more frequently but allows "only" 1000 free queries per day. Not enough while under fierce attack...
I'm monitoring the site of IPdeny to look at their update frequency (little info on the site) to see if that is higher.
Didn't check out info on RIPE yet, but i'll look at them too.

BTW: I found that 2 CC's in iplist-exclusions.conf DO have an IPv4 zone (on IPdeny), AQ_ipv4 (4 /24) & UM_ipv4 (1 /21). No ipv6 though.. I checked the other exclusions also but these are the only ones.
Greetz!!

[friendly-bits]

Hi @QuaxEros, so far I've seen a couple of reports that some IP ranges missing from the ipdendy database were included in the MaxMind database. Here we have the opposite example. I suppose you could report this to MaxMind directly.

As to check-ip-in-source.sh, this script was originally intended as a way to make a quick check before installing geoip-shell. Currently it expects to be called without root permissions, hence it is not trying to read the config from the config file (which stores MaxMind credentials and requires root permissions). I'll look into adding support for that script to run in privileged mode and read the credentials from the config file.

I'll also look into adding support to store the complete MaxMind database locally, so users could avoid re-fetching the database for each check (which quickly runs into MaxMind download limits, especially with the free license).

Thanks for the report about the exclusions file. I'll look into updating it.

In the meantime, I released geoip-shell v0.6.9 which adds support for local IP lists. This was requested by another user. Maybe this feature will come handy to you as well.

[friendly-bits]

Hi @QuaxEros, how are you doing?

I've made a couple of geoip-shell releases since our last communication. These feature a few bugfixes (an important one is a bugfix in IP calculation/aggregation code which could have affected local IPs, trusted IPs, local IP lists and some other things), support for storing the MaxMind database permanently (via the command geoip-shell configure -K true) and support in check-ip-in-source.sh for using previously configured MaxMind credentials (requires to run check-ip-in-source.sh as root).

As to iplist-exclusions.conf. I checked just now and AQ_ipv4 seems to not exist on ipdeny. At least that's what the fetch script says when trying to fetch it. I didn't check further - there is a chance that it's a bug in the fetch script but I doubt so. As to UM_ipv4, indeed it does get fetched correctly from ipdeny, however not from RIPE. Which is why I included it in the exclusions file. Ideally, there should be a mechanism for granular exclusions based on iplist source but this is currently not implemented, and as long as this is so, the current solution of excluding tiny iplists which are only available from some sources but not others seems optimal. If you think it is really important to handle these tiny iplists for each source individually, I can look into implementing this.