Welcome Serena Central users! CLICK HERE
The migration of the Serena Central community is currently underway. Be sure to read THIS MESSAGE to get your new login set up to access your account.
ScorpionSting Absent Member.
Absent Member.
2323 views

Risk Geolocation GeoIP

Just a plug for an Idea to vote on: https://ideas.microfocus.com/MFI/access-manager/Idea/Detail/14580

As GeoIP1 has been sunset (meaning Cool Solutions that relied on GeoIP1 DB's no longer work), the GeoIP2 offers an API through MaxMind

Visit my Website for links to Cool Solution articles.
0 Likes
28 Replies
AutomaticReply Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

ScorpionSting,

It appears that in the past few days you have not received a response to your
posting. That concerns us, and has triggered this automated reply.

These forums are peer-to-peer, best effort, volunteer run and that if your issue
is urgent or not getting a response, you might try one of the following options:

- Visit https://www.microfocus.com/support-and-services and search the knowledgebase and/or check
all the other self support options and support programs available.
- Open a service request: https://www.microfocus.com/support
- You could also try posting your message again. Make sure it is posted in the
correct newsgroup. (http://forums.microfocus.com)
- You might consider hiring a local partner to assist you.
https://www.partnernetprogram.com/partnerfinder/find.html

Be sure to read the forum FAQ about what to expect in the way of responses:
http://forums.microfocus.com/faq.php

Sometimes this automatic posting will alert someone that can respond.

If this is a reply to a duplicate posting or otherwise posted in error, please
ignore and accept our apologies and rest assured we will issue a stern reprimand
to our posting bot.

Good luck!

Your Micro Focus Forums Team
http://forums.microfocus.com



0 Likes
matt4 Honored Contributor.
Honored Contributor.

Re: Risk Geolocation GeoIP

ScorpionSting;2493302 wrote:
Just a plug for an Idea to vote on: https://ideas.microfocus.com/MFI/access-manager/Idea/Detail/14580

As GeoIP1 has been sunset (meaning Cool Solutions that relied on GeoIP1 DB's no longer work), the GeoIP2 offers an API through MaxMind



Has anyone created a new class in the meantime to work with the new MaxMind Geo2 DB?
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

matt;2493786 wrote:
Has anyone created a new class in the meantime to work with the new MaxMind Geo2 DB?


No. I pinged cstumula on his Cool Solution, but he hasn't responded.

Visit my Website for links to Cool Solution articles.
0 Likes
matt4 Honored Contributor.
Honored Contributor.

Re: Risk Geolocation GeoIP

ScorpionSting;2493787 wrote:
No. I pinged cstumula on his Cool Solution, but he hasn't responded.



Do you know if there are any other geo location lists anyone has tried?

I wonder what the Neustar subscription costs? I went to their web site, but I'm not sure exactly what service there I would need to inquire about.

Matt
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

matt;2493852 wrote:
Do you know if there are any other geo location lists anyone has tried?

I wonder what the Neustar subscription costs? I went to their web site, but I'm not sure exactly what service there I would need to inquire about.

Matt


I'm not aware of any...

You'd be after the IP Intelligence product.

Visit my Website for links to Cool Solution articles.
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

ScorpionSting;2493787 wrote:
No. I pinged cstumula on his Cool Solution, but he hasn't responded.


I've taken his java code and adjusted it for:
* better error handling (i.e. metro code is not for everywhere)
* added some other fields for the Risk DB that were missing
* easy search of /var/log/microfocus/idp/tomcat/catalina.out for "MaxMindLocalDB"


package com.netiq.custom.risk.core.geoloc.providers;

import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.util.Properties;

import com.maxmind.geoip2.*;
import com.maxmind.db.*;
import com.maxmind.geoip2.model.*;
import com.maxmind.geoip2.record.*;
import com.maxmind.geoip2.exception.GeoIp2Exception;

import com.novell.nam.nidp.risk.core.geoloc.AbstractProvider;
import com.novell.nam.nidp.risk.core.geoloc.exception.GeoLocException;
import com.novell.nam.nidp.risk.core.geoloc.model.GeoLocBean;
import com.novell.nam.nidp.risk.logging.RiskLog;

public class MaxMindLocalDB extends AbstractProvider
{
String m_CityDBFile = null;
Properties m_Props = null;
String m_OldCityDBFile = null;
static private DatabaseReader dbReader;

public MaxMindLocalDB(Properties props)
{
super(props);
m_Props = props;
}

@Override
public void init(Properties props)
{
m_CityDBFile = props.getProperty("citydbfile");
RiskLog.debug("MaxMindLocalDB: DB file path " + m_CityDBFile);
}

@Override
public GeoLocBean readGeoLocInfo(InetAddress ipAddress) throws GeoLocException
{
boolean createLookupService = false;
GeoLocBean geoLocBean = new GeoLocBean();
try
{
RiskLog.debug("MaxMindLocalDB: IPAddress " + ipAddress);
m_CityDBFile = m_Props.getProperty("citydbfile");
if (m_OldCityDBFile == null || (!m_OldCityDBFile.equalsIgnoreCase(m_CityDBFile)))
{
m_OldCityDBFile = m_CityDBFile;
createLookupService = true;
RiskLog.debug("MaxMindLocalDB: DB new file " + m_CityDBFile + " old file " + m_OldCityDBFile + " createLookservice instance " + createLookupService);
}
RiskLog.debug("MaxMindLocalDB: DB file " + m_CityDBFile );
if (createLookupService || dbReader == null)
{
File database = new File(m_CityDBFile);
dbReader = new DatabaseReader.Builder(database).withCache(new CHMCache()).build();
RiskLog.debug("MaxMindLocalDB: Lookup service instance created");
}
try
{
CityResponse response = dbReader.city(ipAddress);
RiskLog.debug("MaxMindLocalDB: CityResponse " + response.toString());
if (response != null)
{
City city = response.getCity();
Country country = response.getCountry();
Location location = response.getLocation();
Postal postal = response.getPostal();
Subdivision subdivision = response.getMostSpecificSubdivision();
String strTrace = "";
String strCity = null;
String strCountry = null;
String strCountryCode = null;
String strMetroCode = null;
String strPostalCode = null;
String strRegionCode = null;
String strRegionName = null;
String strState = null;
String strStateCode = null;
String strTimeZone = null;
try
{
strCity = city.getName();
strTrace += "\tCity: " + strCity.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strCountry = country.getName();
strTrace += "\tCountry: " + strCountry.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strCountryCode = country.getIsoCode();
strTrace += "\tCountryCode: " + strCountryCode.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strMetroCode = location.getMetroCode().toString();
strTrace += "\tMetroCode: " + strMetroCode.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strPostalCode = postal.getCode();
strTrace += "\tPostalCode: " + strPostalCode.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strRegionCode = subdivision.getIsoCode();
strTrace += "\tRegionCode: " + strRegionCode.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strRegionName = subdivision.getName();
strTrace += "\tRegionName: " + strRegionName.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strState = subdivision.getName();
strTrace += "\tState: " + strState.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strStateCode = subdivision.getIsoCode();
strTrace += "\tStateCode: " + strStateCode.toString() + "\n";
}
catch (RuntimeException e) {}
try
{
strTimeZone = location.getTimeZone();
strTrace += "\tTimeZone: " + strTimeZone.toString() + "\n";
}
catch (RuntimeException e) {}
RiskLog.debug("MaxMindLocalDB:\n" + strTrace);
geoLocBean.setAreaCode(null);
geoLocBean.setCity((strCity != null) ? strCity.toLowerCase() : null);
geoLocBean.setCountry((strCountry != null) ? strCountry.toLowerCase() : null);
geoLocBean.setCountryCode((strCountryCode != null) ? strCountryCode.toLowerCase() : null);
geoLocBean.setMetroCode((strMetroCode != null) ? strMetroCode.toString() : null);
geoLocBean.setOrganization(null);
geoLocBean.setPostalCode((strPostalCode != null) ? strPostalCode : null);
geoLocBean.setRegionCode((strRegionCode != null) ? strRegionCode.toLowerCase() : null);
geoLocBean.setRegionName((strRegionName != null) ? strRegionName.toLowerCase() : null);
geoLocBean.setState((strState != null) ? strState.toLowerCase() : null);
geoLocBean.setStateCode((strStateCode != null) ? strStateCode.toLowerCase() : null);
geoLocBean.setTimeZone((strTimeZone != null) ? strTimeZone : null);
return geoLocBean;
}
}
catch (IOException e)
{
return null;
}
catch (GeoIp2Exception e)
{
return null;
}
}
catch (IOException e)
{
e.printStackTrace();
throw new GeoLocException(e);
}
return null;
}
}


Gives me the output like:


<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: AbstractProvider.<init>
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: DB file path /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: IPAddress /x.x.x.x </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: DB new file /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb old file /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb createLookservice instance true </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: DB file /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: Lookup service instance created </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB: CityResponse com.maxmind.geoip2.model.CityResponse [ {"city":{"geoname_id":2147714,"names":{"de":"Sydney","ru":"Сидней","pt-BR":"Sydney","ja":"シドニー","en":"Sydney","fr":"Sydney","zh-CN":"悉尼","es":"Sídney"}},"continent":{"code":"OC","geoname_id":6255151,"names":{"de":"Ozeanien","ru":"Океания","pt-BR":"Oceania","ja":"オセアニア","en":"Oceania","fr":"Océanie","zh-CN":"大洋洲","es":"Oceanía"}},"country":{"geoname_id":2077456,"is_in_european_union":false,"iso_code":"AU","names":{"de":"Australien","ru":"Австралия","pt-BR":"Austrália","ja":"オーストラリア","en":"Australia","fr":"Australie","zh-CN":"澳大利亚","es":"Australia"}},"location":{"accuracy_radius":1000,"latitude":-33.8678,"longitude":151.2073,"time_zone":"Australia/Sydney"},"maxmind":{},"postal":{"code":"2001"},"registered_country":{"geoname_id":2077456,"is_in_european_union":false,"iso_code":"AU","names":{"de":"Australien","ru":"Австралия","pt-BR":"Austrália","ja":"オーストラリア","en":"Australia","fr":"Australie","zh-CN":"澳大利亚","es":"Australia"}},"represented_country":{"is_in_european_union":false},"subdivisions":[{"geoname_id":2155400,"iso_code":"NSW","names":{"en":"New South Wales","ru":"Новый Южный Уэльс","fr":"Nouvelle-Galles du Sud","pt-BR":"Nova Gales do Sul"}}],"traits":{"ip_address":"x.x.x.x","is_anonymous":false,"is_anonymous_proxy":false,"is_anonymous_vpn":false,"is_hosting_provider":false,"is_legitimate_proxy":false,"is_public_proxy":false,"is_satellite_provider":false,"is_tor_exit_node":false}} ] </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.validateGeoLocation
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
MaxMindLocalDB:
City: Sydney
Country: Australia
CountryCode: AU
PostalCode: 2001
RegionCode: NSW
RegionName: New South Wales
State: New South Wales
StateCode: NSW
TimeZone: Australia/Sydney
</amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: GeoLocation.evaluate
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
GeoLocation Bean: [ country: australia,countryCode: au,city: sydney,timeZone: Australia/Sydney,state: new south wales,stateCode: nsw,areaCode: null,organization: null,postalCode: 2001,metroCode: null,regionCode: nsw,regionName: new south wales,annonymous: false,privateIPAddress: false,AdditionaParameters:null ] </amLogEntry>

<amLogEntry> 2019-03-18T00:47:25Z DEBUG NIDS Application:
Method: ComplexRiskRule.evaluate
Thread: https-jsse-nio-x.x.x.x-8443-exec-4
OR: result: true = true || true </amLogEntry>

Visit my Website for links to Cool Solution articles.
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

I've tickled this code a little more.

MaxMindLocalDB.zip

Includes GeoLocation parameters:


  1. citydbfile = /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb
  2. asndbfile = /var/opt/novell/novlwww/GeoIP/GeoLite2-ASN/GeoLite2-ASN.mmdb


    Either or both can be set. ASN DB allows setting of the Organisation value.


    There are also a couple of scripts:


    • geoip-update.sh

      • Can be cron'ed to download the updates to the DB's automatically to above location (and create location, etc.) after every first Tuesday of the month (Maxmind's update frequency)
      • 0 */6 * * * root /var/opt/novell/novlwww/get-geoip.sh >/dev/null 2>&1


    • max-maxmind.sh

      • Will download Azulu JDK to /var/opt/novell/novlwww/ and set JAVA_HOME & PATH for the script (Azulu Version zulu8.33.0.1-jdk8.0.192 to match NAM 4.4.4)
      • Will check for geoip2 and maxmind jar's in lib directory
      • Will build MaxMindLocalDB.jar and automatically copy it to lib directory, fixing permissions (manual set to systemctl restart novell-idp)



Visit my Website for links to Cool Solution articles.
0 Likes
Highlighted
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

ScorpionSting;2496982 wrote:
I've tickled this code a little more.

MaxMindLocalDB.zip

Includes GeoLocation parameters:


  1. citydbfile = /var/opt/novell/novlwww/GeoIP/GeoLite2-City/GeoLite2-City.mmdb
  2. asndbfile = /var/opt/novell/novlwww/GeoIP/GeoLite2-ASN/GeoLite2-ASN.mmdb


    Either or both can be set. ASN DB allows setting of the Organisation value.


    There are also a couple of scripts:


    • geoip-update.sh

      • Can be cron'ed to download the updates to the DB's automatically to above location (and create location, etc.) after every first Tuesday of the month (Maxmind's update frequency)
      • 0 */6 * * * root /var/opt/novell/novlwww/get-geoip.sh >/dev/null 2>&1


    • max-maxmind.sh

      • Will download Azulu JDK to /var/opt/novell/novlwww/ and set JAVA_HOME & PATH for the script (Azulu Version zulu8.33.0.1-jdk8.0.192 to match NAM 4.4.4)
      • Will check for geoip2 and maxmind jar's in lib directory
      • Will build MaxMindLocalDB.jar and automatically copy it to lib directory, fixing permissions (manual set to systemctl restart novell-idp)





    Appears to be some bugs in the NAM Risk code.... Even though the GeoLocation Bean accepts the additional parameters, it appears that some are not being written to the DB's usrtransaction (i.e. organization, ip(wtf?!), anonymous, regioncode, additional properties)

Visit my Website for links to Cool Solution articles.
0 Likes
jrmhscht Super Contributor.
Super Contributor.

Re: Risk Geolocation GeoIP

Hello,
We updated this to use maxmind v2. I've been meaning to post it but haven't got to it yet. I'll see what I can do in the next few days.
Jeremiah.
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

jrmhscht;2494006 wrote:
Hello,
We updated this to use maxmind v2. I've been meaning to post it but haven't got to it yet. I'll see what I can do in the next few days.
Jeremiah.


Thanks, I see it is now up on your cool solution. Will give it a go over the next few days.

Visit my Website for links to Cool Solution articles.
0 Likes
jrmhscht Super Contributor.
Super Contributor.

Re: Risk Geolocation GeoIP

I am not the author of the cool solution (cstumula), but we did make an updated copy for maxmind v2. If his is working, I probably won't post ours. We did add a json file that overrides specific IP addresses if anyone would like that feature.
0 Likes
matt4 Honored Contributor.
Honored Contributor.

Re: Risk Geolocation GeoIP

jrmhscht;2494054 wrote:
I am not the author of the cool solution (cstumula), but we did make an updated copy for maxmind v2. If his is working, I probably won't post ours. We did add a json file that overrides specific IP addresses if anyone would like that feature.


I tested Chandu's new one and it works! You have to compile it yourself and I noticed the first line has a typo:

package com.netiq.custom.risk.core.geloc.providers;

should be:

package com.netiq.custom.risk.core.geoloc.providers;



I'd be interested in your version that allows you to exclude IP addresses. One issue with the Cool Solution version is that if it doesn't find the IP address in the MaxMind DB (e.g. a private IP) it throws an exception. I got around this with a separate rule to check for internal networks. But I'm all about options!

Matt
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

matt;2494221 wrote:
I tested Chandu's new one and it works! You have to compile it yourself and I noticed the first line has a typo:

package com.netiq.custom.risk.core.geloc.providers;

should be:

package com.netiq.custom.risk.core.geoloc.providers;



I'd be interested in your version that allows you to exclude IP addresses. One issue with the Cool Solution version is that if it doesn't find the IP address in the MaxMind DB (e.g. a private IP) it throws an exception. I got around this with a separate rule to check for internal networks. But I'm all about options!

Matt


Good spot on the typo, that'll probably explain the error I'm seeing:


2019-01-22T23:11:48Z SEVERE NIDS Application: com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDBException message: “com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDB”
WebappClassLoaderBase.java, Line: 1309, Method: loadClass
WebappClassLoaderBase.java, Line: 1137, Method: loadClass
Class.java, Line: -2, Method: forName0
Class.java, Line: 264, Method: forName
GeoLocationFactory.java, Line: 89, Method: getProviderInstance

Visit my Website for links to Cool Solution articles.
0 Likes
ScorpionSting Absent Member.
Absent Member.

Re: Risk Geolocation GeoIP

ScorpionSting;2494222 wrote:
Good spot on the typo, that'll probably explain the error I'm seeing:


2019-01-22T23:11:48Z SEVERE NIDS Application: com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDBException message: “com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDB”
WebappClassLoaderBase.java, Line: 1309, Method: loadClass
WebappClassLoaderBase.java, Line: 1137, Method: loadClass
Class.java, Line: -2, Method: forName0
Class.java, Line: 264, Method: forName
GeoLocationFactory.java, Line: 89, Method: getProviderInstance


Nope, wasn't the problem....


<amLogEntry> 2019-01-24T02:27:50Z SEVERE NIDS Application: com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDBException message: "com.netiq.custom.risk.core.geoloc.providers.MaxMindLocalDB"
WebappClassLoaderBase.java, Line: 1309, Method: loadClass
WebappClassLoaderBase.java, Line: 1137, Method: loadClass
Class.java, Line: -2, Method: forName0
Class.java, Line: 264, Method: forName
GeoLocationFactory.java, Line: 89, Method: getProviderInstance
</amLogEntry>


I'm using this script to compile https://drive.google.com/open?id=1OrxY0DrgrTB9kmBlPDb-VzZIIgopDNub

Visit my Website for links to Cool Solution articles.
0 Likes
The opinions expressed above are the personal opinions of the authors, not of Micro Focus. By using this site, you accept the Terms of Use and Rules of Participation. Certain versions of content ("Material") accessible here may contain branding from Hewlett-Packard Company (now HP Inc.) and Hewlett Packard Enterprise Company. As of September 1, 2017, the Material is now offered by Micro Focus, a separately owned and operated company. Any reference to the HP and Hewlett Packard Enterprise/HPE marks is historical in nature, and the HP and Hewlett Packard Enterprise/HPE marks are the property of their respective owners.